[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:27 dist-test-slave-ss4n krb5kdc[18203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:27 dist-test-slave-ss4n krb5kdc[18203](info): set up 2 sockets
Aug 12 02:42:27 dist-test-slave-ss4n krb5kdc[18203](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 02:42:28.885869 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:42:29 dist-test-slave-ss4n krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:42:29.106580 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:33049
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:33049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:29.454378 18219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:29.454702 18219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:29.454876 18219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:29.462699 18219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:29.462833 18219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:29.462908 18219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:29.463003 18219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:29.463092 18219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:29.473409 18219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:33049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:33049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:29.474885 18219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:29.476966 18219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:29.491285 18225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:29.491295 18226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:29.491200 18228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:29.492208 18219 server_base.cc:1034] running on GCE node
I20260812 02:42:29.493135 18219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:29.494894 18219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:29.496176 18219 hybrid_clock.cc:648] HybridClock initialized: now 1786502549496145 us; error 129 us; skew 500 ppm
I20260812 02:42:29.496646 18219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:29 dist-test-slave-ss4n krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502549, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:29.500474 18219 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:42:29.502285 18219 webserver.cc:460] Webserver started at http://127.17.196.254:46143/ using document root <none> and password file <none>
I20260812 02:42:29.503118 18219 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:29.503283 18219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:29.503639 18219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:29.507128 18219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5"
format_stamp: "Formatted at 2026-08-12 02:42:29 on dist-test-slave-ss4n"
server_key: "cb0714a81973cfcd6fff53907f235cbb"
server_key_iv: "cbfbd21b2b0d540b874451e294840cea"
server_key_version: "encryptionkey@0"
I20260812 02:42:29.507975 18219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5"
format_stamp: "Formatted at 2026-08-12 02:42:29 on dist-test-slave-ss4n"
server_key: "cb0714a81973cfcd6fff53907f235cbb"
server_key_iv: "cbfbd21b2b0d540b874451e294840cea"
server_key_version: "encryptionkey@0"
I20260812 02:42:29.513975 18219 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 02:42:29.518239 18236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:29.519801 18219 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:42:29.520017 18219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5"
format_stamp: "Formatted at 2026-08-12 02:42:29 on dist-test-slave-ss4n"
server_key: "cb0714a81973cfcd6fff53907f235cbb"
server_key_iv: "cbfbd21b2b0d540b874451e294840cea"
server_key_version: "encryptionkey@0"
I20260812 02:42:29.520263 18219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:29.554773 18219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:29.560074 18219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:29.560451 18219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:29.582504 18219 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:33049
I20260812 02:42:29.582536 18290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:33049 every 8 connection(s)
I20260812 02:42:29.584127 18219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:29.588304 18291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:29.589744 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 18219
I20260812 02:42:29.590112 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:42:29.592804 18195 external_mini_cluster.cc:1428] Setting key e12d3e823359e5e745d579ba55097691
I20260812 02:42:29.602622 18291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5: Bootstrap starting.
I20260812 02:42:29.605876 18291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:29.606976 18291 log.cc:824] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:29.610332 18291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5: No bootstrap required, opened a new log
I20260812 02:42:29.620797 18291 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33049 } }
I20260812 02:42:29.621366 18291 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:29.621518 18291 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1dd470acfca4a76950b0ff3f4e3e0a5, State: Initialized, Role: FOLLOWER
I20260812 02:42:29.622349 18291 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33049 } }
I20260812 02:42:29.622624 18291 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:29.622808 18291 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:29.623041 18291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:29.625573 18291 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33049 } }
I20260812 02:42:29.626148 18291 leader_election.cc:302] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1dd470acfca4a76950b0ff3f4e3e0a5; no voters:
Aug 12 02:42:29 dist-test-slave-ss4n krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:29.626986 18291 leader_election.cc:288] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:29.627115 18296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:29.629030 18296 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [term 1 LEADER]: Becoming Leader. State: Replica: f1dd470acfca4a76950b0ff3f4e3e0a5, State: Running, Role: LEADER
I20260812 02:42:29.629654 18291 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:29.629793 18296 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33049 } }
I20260812 02:42:29.637065 18297 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33049 } } }
I20260812 02:42:29.637696 18297 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:29.642920 18298 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1dd470acfca4a76950b0ff3f4e3e0a5. Latest consensus state: current_term: 1 leader_uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1dd470acfca4a76950b0ff3f4e3e0a5" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33049 } } }
I20260812 02:42:29.643252 18298 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:29.644687 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:29.598324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58546 (local address 127.17.196.254:33049)
0812 02:42:29.599370 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:29.599386 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:29.600849 (+ 1463us) server_negotiation.cc:407] Connection header received
0812 02:42:29.610245 (+ 9396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:29.610270 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:29.610740 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:29.611114 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:29.615361 (+ 4247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:29.616890 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:29.619571 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:29.620322 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:29.620547 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:29.628877 (+ 8330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:29.628893 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:29.628917 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:29.628991 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:29.635190 (+ 6199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:29.636509 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:29.636518 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:29.636527 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:29.636772 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:29.637454 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:29.637459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:29.637462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:29.638675 (+ 1213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:29.642743 (+ 4068us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:29.643032 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 02:42:29.643419 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":491,"threads_started":1}
W20260812 02:42:29.648025 18314 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:42:29.648175 18314 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:42:29.648296 18315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:29.650633 18315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:29.660236 18315 catalog_manager.cc:1347] Generated new cluster ID: cba12e2c09f84a1f8a2fee3637e8ca09
I20260812 02:42:29.660426 18315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:29.681311 18315 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:29.682765 18315 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:29.695019 18315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1dd470acfca4a76950b0ff3f4e3e0a5: Generated new TSK 0
I20260812 02:42:29.696223 18315 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:42:29.749248 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33049
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:30.111850 18321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:30.112144 18321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:30.112244 18321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:30.119800 18321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:30.119927 18321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:30.120047 18321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:42:30.130482 18321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33049
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:30.131886 18321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:30.133400 18321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:30.148275 18332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:30.148303 18329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:30.149044 18328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:30.149802 18321 server_base.cc:1034] running on GCE node
I20260812 02:42:30.150454 18321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:30.151571 18321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:30.152841 18321 hybrid_clock.cc:648] HybridClock initialized: now 1786502550152744 us; error 98 us; skew 500 ppm
I20260812 02:42:30.153218 18321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:30 dist-test-slave-ss4n krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502550, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:30.161434 18321 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:42:30.163468 18321 webserver.cc:460] Webserver started at http://127.17.196.193:40901/ using document root <none> and password file <none>
I20260812 02:42:30.164254 18321 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:30.164412 18321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:30.164773 18321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:30.168093 18321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "6fc395bbb411434aaa17d09b0ae55d2c"
format_stamp: "Formatted at 2026-08-12 02:42:30 on dist-test-slave-ss4n"
server_key: "4b30a7640dd65a7fd72d30c599503e84"
server_key_iv: "1b0e8cf977dfc40af53cf3964303cb26"
server_key_version: "encryptionkey@0"
I20260812 02:42:30.169034 18321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "6fc395bbb411434aaa17d09b0ae55d2c"
format_stamp: "Formatted at 2026-08-12 02:42:30 on dist-test-slave-ss4n"
server_key: "4b30a7640dd65a7fd72d30c599503e84"
server_key_iv: "1b0e8cf977dfc40af53cf3964303cb26"
server_key_version: "encryptionkey@0"
I20260812 02:42:30.175083 18321 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 02:42:30.179039 18339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:30.180964 18321 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:42:30.181177 18321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "6fc395bbb411434aaa17d09b0ae55d2c"
format_stamp: "Formatted at 2026-08-12 02:42:30 on dist-test-slave-ss4n"
server_key: "4b30a7640dd65a7fd72d30c599503e84"
server_key_iv: "1b0e8cf977dfc40af53cf3964303cb26"
server_key_version: "encryptionkey@0"
I20260812 02:42:30.181407 18321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:30.198457 18321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:30.203313 18321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:30.203660 18321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:30.204931 18321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:30.206516 18321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:30.206624 18321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:30.206780 18321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:30.206861 18321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:30.247006 18321 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:38447
I20260812 02:42:30.247068 18468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:38447 every 8 connection(s)
I20260812 02:42:30.248481 18321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:42:30.252295 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 18321
I20260812 02:42:30.252558 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:42:30.253304 18195 external_mini_cluster.cc:1428] Setting key 611a8d4e27fc7055fd071aefb37a14ae
Aug 12 02:42:30 dist-test-slave-ss4n krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502550, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:30.273742 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:30.251343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:55233 (local address 127.17.196.254:33049)
0812 02:42:30.252139 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:30.252147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:30.253177 (+ 1030us) server_negotiation.cc:407] Connection header received
0812 02:42:30.255265 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:30.255269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:30.255331 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:30.255454 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:30.256994 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:30.258187 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:30.260359 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:30.261090 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:30.261240 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:30.266522 (+ 5282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:30.266528 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:30.266532 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:30.266573 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:30.269432 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:30.270599 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:30.270604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:30.270606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:30.270737 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:30.271351 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:30.271356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:30.271359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:30.271855 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:30.272005 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:30.273083 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0812 02:42:30.273293 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":382,"threads_started":1}
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
I20260812 02:42:30.275180 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:30.252084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33049 (local address 127.17.196.193:55233)
0812 02:42:30.252963 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:30.253026 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:30.254968 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:30.255695 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:30.255704 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:30.256240 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:30.256814 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:30.256831 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:30.258641 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:30.258646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:30.260173 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:30.260188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:30.261338 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:30.261343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:30.261737 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:30.262612 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:30.262639 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:30.266172 (+ 3533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:30.269678 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:30.269691 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:30.269718 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:30.270451 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:30.270868 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:30.270876 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:30.270882 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:30.271187 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:30.272020 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:30.272030 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:30.272577 (+ 547us) client_negotiation.cc:769] Sending connection context
0812 02:42:30.272943 (+ 366us) client_negotiation.cc:240] Negotiation successful
0812 02:42:30.273657 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":369,"threads_started":1}
I20260812 02:42:30.277406 18469 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
I20260812 02:42:30.277951 18469 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:30.278889 18469 heartbeater.cc:511] Master 127.17.196.254:33049 requested a full tablet report, sending...
I20260812 02:42:30.281890 18254 ts_manager.cc:194] Registered new tserver with Master: 6fc395bbb411434aaa17d09b0ae55d2c (127.17.196.193:38447)
I20260812 02:42:30.284377 18254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:55233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:42:30.309262 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33049
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:30.668640 18475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:30.668964 18475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:30.669135 18475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:30.676736 18475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:30.676880 18475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:30.677073 18475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:42:30.687809 18475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33049
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:30.689275 18475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:30.690758 18475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:30.704797 18483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:30.704748 18480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:30.704830 18481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:30.705925 18475 server_base.cc:1034] running on GCE node
I20260812 02:42:30.706648 18475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:30.707749 18475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:30.709008 18475 hybrid_clock.cc:648] HybridClock initialized: now 1786502550708957 us; error 46 us; skew 500 ppm
I20260812 02:42:30.709355 18475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:30 dist-test-slave-ss4n krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502550, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:30.712811 18475 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:42:30.714694 18475 webserver.cc:460] Webserver started at http://127.17.196.194:34775/ using document root <none> and password file <none>
I20260812 02:42:30.715462 18475 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:30.715662 18475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:30.716004 18475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:30.719745 18475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "7a6ccd5e65384f748644fd750ff707cb"
format_stamp: "Formatted at 2026-08-12 02:42:30 on dist-test-slave-ss4n"
server_key: "7adff9b25c02c50be0d750d0fdfd016f"
server_key_iv: "81202b4441ae2298c852c032ca9924df"
server_key_version: "encryptionkey@0"
I20260812 02:42:30.720633 18475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "7a6ccd5e65384f748644fd750ff707cb"
format_stamp: "Formatted at 2026-08-12 02:42:30 on dist-test-slave-ss4n"
server_key: "7adff9b25c02c50be0d750d0fdfd016f"
server_key_iv: "81202b4441ae2298c852c032ca9924df"
server_key_version: "encryptionkey@0"
I20260812 02:42:30.726939 18475 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 02:42:30.731101 18491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:30.732782 18475 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:42:30.733028 18475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "7a6ccd5e65384f748644fd750ff707cb"
format_stamp: "Formatted at 2026-08-12 02:42:30 on dist-test-slave-ss4n"
server_key: "7adff9b25c02c50be0d750d0fdfd016f"
server_key_iv: "81202b4441ae2298c852c032ca9924df"
server_key_version: "encryptionkey@0"
I20260812 02:42:30.733282 18475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:30.748241 18475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:30.753101 18475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:30.753435 18475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:30.754668 18475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:30.756100 18475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:30.756212 18475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:30.756376 18475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:30.756457 18475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:30.795931 18475 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:34269
I20260812 02:42:30.795993 18615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:34269 every 8 connection(s)
I20260812 02:42:30.797398 18475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:42:30.801769 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 18475
I20260812 02:42:30.801985 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:42:30.802515 18195 external_mini_cluster.cc:1428] Setting key 50f5d3987628ef21cafd7afad7d72b45
Aug 12 02:42:30 dist-test-slave-ss4n krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502550, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:30.824926 18619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:30.800262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:36053 (local address 127.17.196.254:33049)
0812 02:42:30.800796 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:30.800800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:30.803096 (+ 2296us) server_negotiation.cc:407] Connection header received
0812 02:42:30.804223 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:30.804228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:30.804311 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:30.804467 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:30.806772 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:30.807514 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:30.810165 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:30.810688 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:30.810797 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:30.815301 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:30.815315 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:30.815324 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:30.815401 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:30.820083 (+ 4682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:30.821400 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:30.821411 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:30.821416 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:30.821659 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:30.822275 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:30.822284 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:30.822290 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:30.822839 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:30.823056 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:30.823945 (+ 889us) server_negotiation.cc:299] Negotiation successful
0812 02:42:30.824258 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":306,"threads_started":1}
I20260812 02:42:30.825819 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:30.800840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33049 (local address 127.17.196.194:36053)
0812 02:42:30.801799 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:30.801839 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:30.803917 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:30.804784 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:30.804796 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:30.805646 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:30.806552 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:30.806584 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:30.807716 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:30.807725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:30.809980 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:30.810001 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:30.810958 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:30.810968 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:30.811272 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:30.812127 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:30.812151 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:30.814935 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:30.820399 (+ 5464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:30.820409 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:30.820425 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:30.821193 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:30.821811 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:30.821815 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:30.821817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:30.822090 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:30.823031 (+ 941us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:30.823038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:30.823453 (+ 415us) client_negotiation.cc:769] Sending connection context
0812 02:42:30.823714 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 02:42:30.824304 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":392,"threads_started":1}
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
I20260812 02:42:30.827652 18617 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33049
I20260812 02:42:30.828071 18617 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:30.829027 18617 heartbeater.cc:511] Master 127.17.196.254:33049 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
I20260812 02:42:30.831116 18254 ts_manager.cc:194] Registered new tserver with Master: 7a6ccd5e65384f748644fd750ff707cb (127.17.196.194:34269)
I20260812 02:42:30.832106 18254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:36053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:42:30.858739 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33049
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:31.212456 18624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:31.212762 18624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:31.212949 18624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:31.220456 18624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:31.220588 18624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:31.220791 18624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:42:31.231623 18624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33049
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:31.233101 18624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:31.234628 18624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:31.249644 18629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:31.249675 18630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:31.249909 18632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:31.251650 18624 server_base.cc:1034] running on GCE node
I20260812 02:42:31.252310 18624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:31.253391 18624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:31.254733 18624 hybrid_clock.cc:648] HybridClock initialized: now 1786502551254665 us; error 59 us; skew 500 ppm
I20260812 02:42:31.255165 18624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:31 dist-test-slave-ss4n krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502551, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:31.259300 18624 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:42:31.261243 18624 webserver.cc:460] Webserver started at http://127.17.196.195:34175/ using document root <none> and password file <none>
I20260812 02:42:31.262112 18624 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:31.262303 18624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:31.262684 18624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:31.266146 18624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "71e1f60a6ee14ea48ea2e6ed7732c193"
format_stamp: "Formatted at 2026-08-12 02:42:31 on dist-test-slave-ss4n"
server_key: "ab8c430864b613a81cd1a6378e66211f"
server_key_iv: "5328bab37376c17efd36762ba645c8be"
server_key_version: "encryptionkey@0"
I20260812 02:42:31.267019 18624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "71e1f60a6ee14ea48ea2e6ed7732c193"
format_stamp: "Formatted at 2026-08-12 02:42:31 on dist-test-slave-ss4n"
server_key: "ab8c430864b613a81cd1a6378e66211f"
server_key_iv: "5328bab37376c17efd36762ba645c8be"
server_key_version: "encryptionkey@0"
I20260812 02:42:31.273303 18624 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 02:42:31.278031 18639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:31.279731 18624 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:42:31.279984 18624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "71e1f60a6ee14ea48ea2e6ed7732c193"
format_stamp: "Formatted at 2026-08-12 02:42:31 on dist-test-slave-ss4n"
server_key: "ab8c430864b613a81cd1a6378e66211f"
server_key_iv: "5328bab37376c17efd36762ba645c8be"
server_key_version: "encryptionkey@0"
I20260812 02:42:31.280349 18624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:31.288882 18469 heartbeater.cc:503] Master 127.17.196.254:33049 was elected leader, sending a full tablet report...
I20260812 02:42:31.329118 18624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:31.334784 18624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:31.335125 18624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:31.336192 18624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:31.337899 18624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:31.338012 18624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:31.338157 18624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:31.338233 18624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:31.376529 18624 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:41607
I20260812 02:42:31.376658 18760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:41607 every 8 connection(s)
I20260812 02:42:31.377964 18624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:42:31.382521 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 18624
I20260812 02:42:31.382742 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:42:31.383339 18195 external_mini_cluster.cc:1428] Setting key 81a669224e9c398236fb8c1da44c0b35
Aug 12 02:42:31 dist-test-slave-ss4n krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502551, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:31.401716 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:31.380994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:33039 (local address 127.17.196.254:33049)
0812 02:42:31.381560 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:31.381567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:31.382826 (+ 1259us) server_negotiation.cc:407] Connection header received
0812 02:42:31.385641 (+ 2815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:31.385646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:31.385723 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:31.385823 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:31.387916 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.388701 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.390433 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.390856 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.391002 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:31.394754 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:31.394760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:31.394764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:31.394799 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:31.397944 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.398831 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.398836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.398838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.398978 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.399558 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.399564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.399567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.399957 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:31.400160 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:31.400966 (+ 806us) server_negotiation.cc:299] Negotiation successful
0812 02:42:31.401235 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":226,"threads_started":1}
I20260812 02:42:31.402762 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:31.381683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33049 (local address 127.17.196.195:33039)
0812 02:42:31.382609 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:31.382675 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:31.385164 (+ 2489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:31.386155 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:31.386168 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:31.387015 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:31.387737 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:31.387757 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.388866 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.388872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:31.390274 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:31.390288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.391121 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.391129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:31.391392 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:31.392163 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:31.392187 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:31.394501 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:31.398144 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:31.398153 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:31.398168 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:31.398668 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:31.399136 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:31.399140 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:31.399143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:31.399436 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:31.400115 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:31.400125 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:31.400542 (+ 417us) client_negotiation.cc:769] Sending connection context
0812 02:42:31.400849 (+ 307us) client_negotiation.cc:240] Negotiation successful
0812 02:42:31.401468 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":362,"threads_started":1}
I20260812 02:42:31.404289 18761 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33049
I20260812 02:42:31.404660 18761 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:31.405419 18761 heartbeater.cc:511] Master 127.17.196.254:33049 requested a full tablet report, sending...
I20260812 02:42:31.407109 18254 ts_manager.cc:194] Registered new tserver with Master: 71e1f60a6ee14ea48ea2e6ed7732c193 (127.17.196.195:41607)
I20260812 02:42:31.407944 18254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:33039
I20260812 02:42:31.413657 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:42:31 dist-test-slave-ss4n krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502551, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:42:31 dist-test-slave-ss4n krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502551, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:42:31.469702 18772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:31.452286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50220 (local address 127.17.196.193:38447)
0812 02:42:31.453141 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:31.453150 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:31.453181 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:42:31.453291 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:31.453299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:31.453628 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:31.453780 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:31.455373 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.456556 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.458016 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.458715 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.458827 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:31.463311 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:31.463327 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:31.463345 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:31.463387 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:31.466626 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.467430 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.467437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.467440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.467582 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.467971 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.467974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.467976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.468370 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:31.468609 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:31.468954 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 02:42:31.469201 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":503,"threads_started":1}
W20260812 02:42:31.471100 18370 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50220
Aug 12 02:42:31 dist-test-slave-ss4n krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:42:31 dist-test-slave-ss4n krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:42:31.523787 18772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:31.509000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50236 (local address 127.17.196.193:38447)
0812 02:42:31.509216 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:31.509225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:31.509709 (+ 484us) server_negotiation.cc:407] Connection header received
0812 02:42:31.509949 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:31.509953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:31.510023 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:31.510137 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:31.511549 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.512281 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.513646 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.514091 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.514236 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:31.518489 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:31.518494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:31.518498 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:31.518539 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:31.521164 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.521943 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.521948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.521950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.522078 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.522529 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.522532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.522534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.522792 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:31.522971 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:31.523335 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 02:42:31.523511 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:42:31.525875 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 18321
I20260812 02:42:31.539590 18772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:31.526891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50252 (local address 127.17.196.193:38447)
0812 02:42:31.527054 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:31.527058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:31.527436 (+ 378us) server_negotiation.cc:407] Connection header received
0812 02:42:31.527545 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:31.527548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:31.527601 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:31.527680 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:31.528906 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.529698 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.531104 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.531586 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.531685 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:31.534206 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:31.534210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:31.534213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:31.534245 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:31.536926 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.537612 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.537616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.537618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.537758 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.538186 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.538189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.538191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.538449 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:31.538628 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:31.539018 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 02:42:31.539242 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 02:42:31.540596 18464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:31.632849 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 18321
I20260812 02:42:31.654021 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 18475
Aug 12 02:42:31 dist-test-slave-ss4n krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:42:31.672171 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:31.655058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50748 (local address 127.17.196.194:34269)
0812 02:42:31.655652 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:31.655660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:31.655686 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:42:31.655750 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:31.655756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:31.655974 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:31.656140 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:31.657600 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.658680 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.660064 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.660659 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.660765 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:31.665677 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:31.665692 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:31.665708 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:31.665747 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:31.668813 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.669638 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.669646 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.669648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.669780 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.670175 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.670179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.670180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.670614 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:31.670862 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:31.671336 (+ 474us) server_negotiation.cc:299] Negotiation successful
0812 02:42:31.671678 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":292,"threads_started":1}
I20260812 02:42:31.673779 18611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:31.761359 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 18475
I20260812 02:42:31.780990 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 18624
Aug 12 02:42:31 dist-test-slave-ss4n krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:42:31.800498 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:31.782121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.17.196.195:41607)
0812 02:42:31.782719 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:31.782728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:31.782759 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:42:31.782826 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:31.782833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:31.783057 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:31.783195 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:31.784912 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.786587 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.788569 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:31.789241 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:31.789423 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:31.794397 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:31.794411 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:31.794428 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:31.794515 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:31.797765 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.798539 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.798546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.798548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.798676 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:31.799020 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:31.799024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:31.799026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:31.799383 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:31.799680 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:31.799934 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 02:42:31.800135 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":270,"threads_started":1}
I20260812 02:42:31.802237 18755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:31.888108 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 18624
I20260812 02:42:31.909925 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 18219
I20260812 02:42:31.911329 18286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:32.028899 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 18219
2026-08-12T02:42:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4414 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:32 dist-test-slave-ss4n krb5kdc[18787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:32 dist-test-slave-ss4n krb5kdc[18787](info): set up 2 sockets
Aug 12 02:42:32 dist-test-slave-ss4n krb5kdc[18787](info): commencing operation
krb5kdc: starting...
W20260812 02:42:33.234802 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.005s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:42:33 dist-test-slave-ss4n krb5kdc[18787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:42:33.466635 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37767
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:37767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:33.808583 18804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:33.808892 18804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:33.809064 18804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:33.816573 18804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:33.816694 18804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:33.816792 18804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:33.816880 18804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:33.816922 18804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:33.827081 18804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:37767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:33.828543 18804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:33.829928 18804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:33.842677 18809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:33.842720 18810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:33.842638 18812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:33.847323 18804 server_base.cc:1034] running on GCE node
I20260812 02:42:33.848359 18804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:33.850040 18804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:33.851387 18804 hybrid_clock.cc:648] HybridClock initialized: now 1786502553851224 us; error 147 us; skew 500 ppm
I20260812 02:42:33.851820 18804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:33 dist-test-slave-ss4n krb5kdc[18787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502553, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:33.855302 18804 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:42:33.857050 18804 webserver.cc:460] Webserver started at http://127.17.196.254:44425/ using document root <none> and password file <none>
I20260812 02:42:33.857967 18804 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:33.858084 18804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:33.858364 18804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:33.861599 18804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "5720297b13eb401c823ec39a7ac2a509"
format_stamp: "Formatted at 2026-08-12 02:42:33 on dist-test-slave-ss4n"
server_key: "70b7747d4097e47773fd59e0bcee4f23"
server_key_iv: "64e3d435f2bc7bf215d6f8bb58c746bd"
server_key_version: "encryptionkey@0"
I20260812 02:42:33.862386 18804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "5720297b13eb401c823ec39a7ac2a509"
format_stamp: "Formatted at 2026-08-12 02:42:33 on dist-test-slave-ss4n"
server_key: "70b7747d4097e47773fd59e0bcee4f23"
server_key_iv: "64e3d435f2bc7bf215d6f8bb58c746bd"
server_key_version: "encryptionkey@0"
I20260812 02:42:33.868114 18804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:42:33.871825 18819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:33.873256 18804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:42:33.873449 18804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "5720297b13eb401c823ec39a7ac2a509"
format_stamp: "Formatted at 2026-08-12 02:42:33 on dist-test-slave-ss4n"
server_key: "70b7747d4097e47773fd59e0bcee4f23"
server_key_iv: "64e3d435f2bc7bf215d6f8bb58c746bd"
server_key_version: "encryptionkey@0"
I20260812 02:42:33.873790 18804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:33.903779 18804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:33.908869 18804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:33.909216 18804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:33.931900 18804 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:37767
I20260812 02:42:33.931903 18877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:37767 every 8 connection(s)
I20260812 02:42:33.933382 18804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:33.937371 18878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:33.939851 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 18804
I20260812 02:42:33.940112 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:42:33.940783 18195 external_mini_cluster.cc:1428] Setting key 5a9d5e576abdce5d59d773ca96c46509
I20260812 02:42:33.949275 18878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509: Bootstrap starting.
I20260812 02:42:33.952579 18878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:33.953733 18878 log.cc:824] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:42:33 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:33.956925 18878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509: No bootstrap required, opened a new log
I20260812 02:42:33.964475 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:33.943031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.17.196.254:37767)
0812 02:42:33.943894 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:33.943907 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:33.943952 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:42:33.945830 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:33.945857 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:33.946329 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:33.946780 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:33.948378 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:33.949784 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:33.951063 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:33.951750 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:33.951958 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:33.956465 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:33.956473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:33.956488 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:33.956528 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:33.959888 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:33.960678 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:33.960692 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:33.960699 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:33.960876 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:33.961286 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:33.961298 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:33.961300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:33.961969 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:33.962348 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:33.962883 (+ 535us) server_negotiation.cc:299] Negotiation successful
0812 02:42:33.963360 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":401,"threads_started":1}
I20260812 02:42:33.965269 18878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5720297b13eb401c823ec39a7ac2a509" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37767 } }
I20260812 02:42:33.965740 18878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:33.965868 18878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5720297b13eb401c823ec39a7ac2a509, State: Initialized, Role: FOLLOWER
I20260812 02:42:33.966596 18878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5720297b13eb401c823ec39a7ac2a509" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37767 } }
I20260812 02:42:33.966879 18878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:33.967031 18878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:33.967221 18878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:33.969946 18878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5720297b13eb401c823ec39a7ac2a509" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37767 } }
I20260812 02:42:33.970439 18878 leader_election.cc:302] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5720297b13eb401c823ec39a7ac2a509; no voters:
I20260812 02:42:33.971105 18878 leader_election.cc:288] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:33.971261 18883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:33.972411 18883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [term 1 LEADER]: Becoming Leader. State: Replica: 5720297b13eb401c823ec39a7ac2a509, State: Running, Role: LEADER
I20260812 02:42:33.973067 18883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5720297b13eb401c823ec39a7ac2a509" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37767 } }
I20260812 02:42:33.973574 18878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:33.976301 18884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5720297b13eb401c823ec39a7ac2a509" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5720297b13eb401c823ec39a7ac2a509" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37767 } } }
I20260812 02:42:33.976290 18885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5720297b13eb401c823ec39a7ac2a509. Latest consensus state: current_term: 1 leader_uuid: "5720297b13eb401c823ec39a7ac2a509" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5720297b13eb401c823ec39a7ac2a509" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37767 } } }
I20260812 02:42:33.976800 18885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:33.976800 18884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:33.977807 18890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:33.983603 18890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:33.994320 18890 catalog_manager.cc:1347] Generated new cluster ID: a83936e6bad1439ebbcc4fe86c332a70
I20260812 02:42:33.994491 18890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:34.012794 18890 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:34.014540 18890 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:34.024575 18890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5720297b13eb401c823ec39a7ac2a509: Generated new TSK 0
I20260812 02:42:34.025645 18890 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:42:34.090750 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37767
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:34.449337 18905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:34.449779 18905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:34.449944 18905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:34.457852 18905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:34.457988 18905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:34.458197 18905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:42:34.468890 18905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:34.470394 18905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:34.471940 18905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:34.487980 18914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:34.487946 18912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:34.487916 18910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:34.489411 18905 server_base.cc:1034] running on GCE node
I20260812 02:42:34.490227 18905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:34.491392 18905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:34.492697 18905 hybrid_clock.cc:648] HybridClock initialized: now 1786502554492636 us; error 94 us; skew 500 ppm
I20260812 02:42:34.493093 18905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:34 dist-test-slave-ss4n krb5kdc[18787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502554, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:34.497154 18905 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:42:34.499447 18905 webserver.cc:460] Webserver started at http://127.17.196.193:34369/ using document root <none> and password file <none>
I20260812 02:42:34.500288 18905 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:34.500447 18905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:34.500841 18905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:34.504443 18905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "ce3dcb5614404ee89647827f758d94a4"
format_stamp: "Formatted at 2026-08-12 02:42:34 on dist-test-slave-ss4n"
server_key: "ae42e59166ba0d429759e10ae8877c5b"
server_key_iv: "3e91611edb958cacfb3fee375ed45a35"
server_key_version: "encryptionkey@0"
I20260812 02:42:34.505371 18905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "ce3dcb5614404ee89647827f758d94a4"
format_stamp: "Formatted at 2026-08-12 02:42:34 on dist-test-slave-ss4n"
server_key: "ae42e59166ba0d429759e10ae8877c5b"
server_key_iv: "3e91611edb958cacfb3fee375ed45a35"
server_key_version: "encryptionkey@0"
I20260812 02:42:34.511756 18905 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:42:34.515992 18923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:34.517827 18905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:42:34.517995 18905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "ce3dcb5614404ee89647827f758d94a4"
format_stamp: "Formatted at 2026-08-12 02:42:34 on dist-test-slave-ss4n"
server_key: "ae42e59166ba0d429759e10ae8877c5b"
server_key_iv: "3e91611edb958cacfb3fee375ed45a35"
server_key_version: "encryptionkey@0"
I20260812 02:42:34.518210 18905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:34.543277 18905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:34.548661 18905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:34.548954 18905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:34.550163 18905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:34.551739 18905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:34.551831 18905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:34.551926 18905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:34.551976 18905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:34.591272 18905 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:35109
I20260812 02:42:34.591370 19043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:35109 every 8 connection(s)
I20260812 02:42:34.592731 18905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:42:34.594411 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 18905
I20260812 02:42:34.594677 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:42:34.595489 18195 external_mini_cluster.cc:1428] Setting key 8468cfbb4c902768bd73cb20c2ad5671
Aug 12 02:42:34 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502554, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:34.616583 19046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:34.595774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:52511 (local address 127.17.196.254:37767)
0812 02:42:34.596546 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:34.596553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:34.597548 (+ 995us) server_negotiation.cc:407] Connection header received
0812 02:42:34.599506 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:34.599515 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:34.599630 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:34.599770 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:34.601399 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:34.602844 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:34.604754 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:34.605219 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:34.605337 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:34.609534 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:34.609543 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:34.609550 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:34.609595 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:34.612775 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:34.613822 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:34.613830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:34.613833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:34.613979 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:34.614522 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:34.614531 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:34.614537 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:34.614882 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:34.615069 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:34.615887 (+ 818us) server_negotiation.cc:299] Negotiation successful
0812 02:42:34.616152 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":460,"threads_started":1}
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
I20260812 02:42:34.617898 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:34.596444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37767 (local address 127.17.196.193:52511)
0812 02:42:34.597336 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:34.597386 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:34.599204 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:34.599968 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:34.599977 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:34.600591 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:34.601197 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:34.601215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:34.603080 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:34.603084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:34.604535 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:34.604559 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:34.605518 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:34.605527 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:34.605907 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:34.606798 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:34.606822 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:34.609216 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:34.613041 (+ 3825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:34.613050 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:34.613067 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:34.613674 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:34.614113 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:34.614118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:34.614124 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:34.614341 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:34.615046 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:34.615056 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:34.615455 (+ 399us) client_negotiation.cc:769] Sending connection context
0812 02:42:34.615758 (+ 303us) client_negotiation.cc:240] Negotiation successful
0812 02:42:34.616400 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":355,"threads_started":1}
I20260812 02:42:34.619681 19044 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37767
I20260812 02:42:34.620124 19044 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
I20260812 02:42:34.621079 19044 heartbeater.cc:511] Master 127.17.196.254:37767 requested a full tablet report, sending...
I20260812 02:42:34.623801 18837 ts_manager.cc:194] Registered new tserver with Master: ce3dcb5614404ee89647827f758d94a4 (127.17.196.193:35109)
I20260812 02:42:34.625414 18837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:52511
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:42:34.651130 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37767
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:35.020515 19050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:35.020903 19050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:35.021096 19050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:35.029131 19050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:35.029330 19050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:35.029557 19050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:42:35.040159 19050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:35.041721 19050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:35.043401 19050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:35.058576 19059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:35.058589 19055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:35.058599 19056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:35.059669 19050 server_base.cc:1034] running on GCE node
I20260812 02:42:35.060352 19050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:35.061470 19050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:35.062821 19050 hybrid_clock.cc:648] HybridClock initialized: now 1786502555062781 us; error 42 us; skew 500 ppm
I20260812 02:42:35.063222 19050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:35 dist-test-slave-ss4n krb5kdc[18787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502555, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:35.066895 19050 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:42:35.068732 19050 webserver.cc:460] Webserver started at http://127.17.196.194:43659/ using document root <none> and password file <none>
I20260812 02:42:35.069581 19050 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:35.069752 19050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:35.070111 19050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:35.073312 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "6fb7836ec0a041fb804644686f9633f9"
format_stamp: "Formatted at 2026-08-12 02:42:35 on dist-test-slave-ss4n"
server_key: "bf48cc5ce5373b08f277dfc6e5692c48"
server_key_iv: "0c3f6c25413366789032a0bcf26a1bd6"
server_key_version: "encryptionkey@0"
I20260812 02:42:35.074333 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "6fb7836ec0a041fb804644686f9633f9"
format_stamp: "Formatted at 2026-08-12 02:42:35 on dist-test-slave-ss4n"
server_key: "bf48cc5ce5373b08f277dfc6e5692c48"
server_key_iv: "0c3f6c25413366789032a0bcf26a1bd6"
server_key_version: "encryptionkey@0"
I20260812 02:42:35.080339 19050 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260812 02:42:35.084704 19068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:35.086283 19050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:42:35.086488 19050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "6fb7836ec0a041fb804644686f9633f9"
format_stamp: "Formatted at 2026-08-12 02:42:35 on dist-test-slave-ss4n"
server_key: "bf48cc5ce5373b08f277dfc6e5692c48"
server_key_iv: "0c3f6c25413366789032a0bcf26a1bd6"
server_key_version: "encryptionkey@0"
I20260812 02:42:35.086725 19050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:35.103762 19050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:35.108433 19050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:35.108745 19050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:35.109892 19050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:35.111356 19050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:35.111461 19050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:35.111617 19050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:35.111696 19050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:35.148274 19050 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:32899
I20260812 02:42:35.148356 19188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:32899 every 8 connection(s)
I20260812 02:42:35.149818 19050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:42:35.155174 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 19050
I20260812 02:42:35.155380 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:42:35.155958 18195 external_mini_cluster.cc:1428] Setting key 9562e676cf1d1122d85df5eccf430662
Aug 12 02:42:35 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502555, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
I20260812 02:42:35.180073 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.152646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:43191 (local address 127.17.196.254:37767)
0812 02:42:35.153152 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:35.153158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:35.154106 (+ 948us) server_negotiation.cc:407] Connection header received
0812 02:42:35.156742 (+ 2636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:35.156748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:35.156851 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:35.156985 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:35.158969 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.159831 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.162583 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.163315 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.163474 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.171836 (+ 8362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:35.171847 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:35.171855 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:35.171908 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:35.176039 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:35.176976 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:35.176983 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:35.176990 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:35.177216 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:35.177822 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:35.177829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:35.177833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:35.178300 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:35.178472 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:35.179490 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0812 02:42:35.179674 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":252,"threads_started":1}
I20260812 02:42:35.181201 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.153050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37767 (local address 127.17.196.194:43191)
0812 02:42:35.153914 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:35.153964 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:35.156382 (+ 2418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:35.157246 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:35.157255 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:35.157877 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:35.158740 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.158773 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.160022 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.160030 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.162377 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.162397 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.163613 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.163623 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.163981 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.168612 (+ 4631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:35.168671 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:35.171480 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:35.176226 (+ 4746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:35.176235 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:35.176250 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:35.176793 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:35.177354 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:35.177357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:35.177360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:35.177679 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:35.178528 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:35.178535 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:35.179006 (+ 471us) client_negotiation.cc:769] Sending connection context
0812 02:42:35.179334 (+ 328us) client_negotiation.cc:240] Negotiation successful
0812 02:42:35.179825 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":285,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
I20260812 02:42:35.183365 19189 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37767
I20260812 02:42:35.183811 19189 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:35.184661 19189 heartbeater.cc:511] Master 127.17.196.254:37767 requested a full tablet report, sending...
I20260812 02:42:35.186560 18837 ts_manager.cc:194] Registered new tserver with Master: 6fb7836ec0a041fb804644686f9633f9 (127.17.196.194:32899)
I20260812 02:42:35.187952 18837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:43191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:42:35.213712 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37767
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:35.563294 19195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:35.563676 19195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:35.563856 19195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:35.571484 19195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:35.571655 19195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:35.571843 19195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:42:35.582537 19195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:35.584127 19195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:35.585690 19195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:35.601671 19200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:35.601647 19201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:35.601823 19203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:35.602548 19195 server_base.cc:1034] running on GCE node
I20260812 02:42:35.603278 19195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:35.604502 19195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:35.605715 19195 hybrid_clock.cc:648] HybridClock initialized: now 1786502555605666 us; error 48 us; skew 500 ppm
I20260812 02:42:35.606109 19195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:35 dist-test-slave-ss4n krb5kdc[18787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502555, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:35.609724 19195 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:42:35.611429 19195 webserver.cc:460] Webserver started at http://127.17.196.195:43399/ using document root <none> and password file <none>
I20260812 02:42:35.612181 19195 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:35.612339 19195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:35.612677 19195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:35.615806 19195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "e13a4dfa42a843dc8429345d27313141"
format_stamp: "Formatted at 2026-08-12 02:42:35 on dist-test-slave-ss4n"
server_key: "5105aa813225eaa57bf1f1209fb5b541"
server_key_iv: "0ba0a64b46f7f443b9fd685d5af28398"
server_key_version: "encryptionkey@0"
I20260812 02:42:35.616642 19195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "e13a4dfa42a843dc8429345d27313141"
format_stamp: "Formatted at 2026-08-12 02:42:35 on dist-test-slave-ss4n"
server_key: "5105aa813225eaa57bf1f1209fb5b541"
server_key_iv: "0ba0a64b46f7f443b9fd685d5af28398"
server_key_version: "encryptionkey@0"
I20260812 02:42:35.622756 19195 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:42:35.627329 19211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:35.628724 19044 heartbeater.cc:503] Master 127.17.196.254:37767 was elected leader, sending a full tablet report...
I20260812 02:42:35.629117 19195 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260812 02:42:35.629326 19195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "e13a4dfa42a843dc8429345d27313141"
format_stamp: "Formatted at 2026-08-12 02:42:35 on dist-test-slave-ss4n"
server_key: "5105aa813225eaa57bf1f1209fb5b541"
server_key_iv: "0ba0a64b46f7f443b9fd685d5af28398"
server_key_version: "encryptionkey@0"
I20260812 02:42:35.629676 19195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:35.646265 19195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:35.652015 19195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:35.652421 19195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:35.653846 19195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:35.655645 19195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:35.655777 19195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:35.655961 19195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:35.656059 19195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:35.696309 19195 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:44167
I20260812 02:42:35.696367 19334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:44167 every 8 connection(s)
I20260812 02:42:35.697885 19195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:42:35.708437 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 19195
I20260812 02:42:35.708626 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:42:35.709115 18195 external_mini_cluster.cc:1428] Setting key 7b2f80ab180fc08f51dbdb0ab59f9f6b
Aug 12 02:42:35 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502555, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:35.723106 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.700896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:56929 (local address 127.17.196.254:37767)
0812 02:42:35.701561 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:35.701566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:35.702595 (+ 1029us) server_negotiation.cc:407] Connection header received
0812 02:42:35.704673 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:35.704680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:35.704804 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:35.704944 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:35.706867 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.708033 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.710293 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.710948 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.711112 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.715910 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:35.715915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:35.715919 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:35.715973 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:35.719113 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:35.720156 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:35.720160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:35.720163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:35.720311 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:35.720805 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:35.720809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:35.720811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:35.721154 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:35.721287 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:35.722453 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0812 02:42:35.722705 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":298,"threads_started":1}
I20260812 02:42:35.724149 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.701371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37767 (local address 127.17.196.195:56929)
0812 02:42:35.702319 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:35.702374 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:35.704357 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:35.705235 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:35.705244 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:35.705942 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:35.706604 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.706624 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.708289 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.708294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.710129 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.710146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.711246 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.711255 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.711627 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.712570 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:35.712598 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:35.715565 (+ 2967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:35.719308 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:35.719318 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:35.719336 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:35.719923 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:35.720429 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:35.720433 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:35.720435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:35.720681 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:35.721360 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:35.721369 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:35.721902 (+ 533us) client_negotiation.cc:769] Sending connection context
0812 02:42:35.722201 (+ 299us) client_negotiation.cc:240] Negotiation successful
0812 02:42:35.722738 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":347,"threads_started":1}
I20260812 02:42:35.725994 19335 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37767
I20260812 02:42:35.726429 19335 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:35.727283 19335 heartbeater.cc:511] Master 127.17.196.254:37767 requested a full tablet report, sending...
I20260812 02:42:35.728937 18837 ts_manager.cc:194] Registered new tserver with Master: e13a4dfa42a843dc8429345d27313141 (127.17.196.195:44167)
I20260812 02:42:35.729936 18837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:56929
I20260812 02:42:35.739266 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:42:35.761454 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.746881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55032 (local address 127.17.196.254:37767)
0812 02:42:35.747099 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:35.747104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:35.747655 (+ 551us) server_negotiation.cc:407] Connection header received
0812 02:42:35.747837 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:35.747842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:35.747918 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:35.748033 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:35.749733 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.750587 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.751974 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.752429 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.752552 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.755317 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:35.755323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:35.755328 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:35.755371 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:35.758563 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:35.759356 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:35.759361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:35.759364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:35.759500 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:35.759988 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:35.759992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:35.759994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:35.760358 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:35.760565 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:35.760938 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 02:42:35.761169 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 02:42:35.769407 18837 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:42:35.776750 18837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:35.806387 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.792526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.17.196.195:44167)
0812 02:42:35.796359 (+ 3833us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:35.796366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:35.796397 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:42:35.796493 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:35.796498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:35.796720 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:35.797285 (+ 565us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:35.797659 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.799572 (+ 1913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.803642 (+ 4070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.804908 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.805183 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.805714 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:35.805812 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 02:42:35.805988 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3683,"thread_start_us":3481,"threads_started":1}
I20260812 02:42:35.806476 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.791967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:44167 (local address 127.0.0.1:47944)
0812 02:42:35.792862 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:35.792913 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:35.793143 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:35.796988 (+ 3845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:35.796996 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:35.797029 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:35.797437 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.797455 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.799748 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.799754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.803397 (+ 3643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.803415 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.805163 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.805170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.805395 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.805454 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 02:42:35.805650 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 02:42:35.805880 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":49,"thread_start_us":869,"threads_started":1}
I20260812 02:42:35.807597 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.792502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:32899 (local address 127.0.0.1:57198)
0812 02:42:35.798087 (+ 5585us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:35.798107 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:35.798265 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:35.799344 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:35.799348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:35.799370 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:35.799748 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.799758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.801332 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.801339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.803920 (+ 2581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.803935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.806440 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.806447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.806658 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.806692 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:42:35.806917 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 02:42:35.807095 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5416,"mutex_wait_us":1,"thread_start_us":5226,"threads_started":1}
I20260812 02:42:35.807641 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.793048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57198 (local address 127.17.196.194:32899)
0812 02:42:35.797462 (+ 4414us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:35.797502 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:35.798412 (+ 910us) server_negotiation.cc:407] Connection header received
0812 02:42:35.798546 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:35.798559 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:35.798838 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:35.799081 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:35.799957 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.801163 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.804136 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.805879 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.806075 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.806249 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:35.807008 (+ 759us) server_negotiation.cc:299] Negotiation successful
0812 02:42:35.807226 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4240,"thread_start_us":4101,"threads_started":1}
I20260812 02:42:35.808383 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.792177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:35109 (local address 127.0.0.1:47570)
0812 02:42:35.794984 (+ 2807us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:35.795019 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:35.795182 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:35.796757 (+ 1575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:35.796769 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:35.796805 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:35.798037 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.798053 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.800193 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.800199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.803198 (+ 2999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:35.803213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.807354 (+ 4141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.807364 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:35.807613 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.807658 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 02:42:35.807762 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 02:42:35.807917 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1535,"mutex_wait_us":342,"spinlock_wait_cycles":1280,"thread_start_us":1343,"threads_started":1}
I20260812 02:42:35.810140 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:35.792915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47570 (local address 127.17.196.193:35109)
0812 02:42:35.795936 (+ 3021us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:35.795961 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:35.796001 (+ 40us) server_negotiation.cc:407] Connection header received
0812 02:42:35.796139 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:35.796152 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:35.796469 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:35.796744 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:35.798319 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.800012 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.803400 (+ 3388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:35.804885 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:35.805136 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:35.805351 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:35.808105 (+ 2754us) server_negotiation.cc:299] Negotiation successful
0812 02:42:35.809776 (+ 1671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2795,"thread_start_us":2595,"threads_started":1}
I20260812 02:42:35.811632 19267 tablet_service.cc:1467] Processing CreateTablet for tablet e41cec91ece94469a9405e88dc9f8d81 (DEFAULT_TABLE table=test-table [id=97991b3b060744ec8dc2e132401a94e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:35.812191 19120 tablet_service.cc:1467] Processing CreateTablet for tablet e41cec91ece94469a9405e88dc9f8d81 (DEFAULT_TABLE table=test-table [id=97991b3b060744ec8dc2e132401a94e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:35.813531 19267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41cec91ece94469a9405e88dc9f8d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:35.813743 19120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41cec91ece94469a9405e88dc9f8d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:35.814007 18975 tablet_service.cc:1467] Processing CreateTablet for tablet e41cec91ece94469a9405e88dc9f8d81 (DEFAULT_TABLE table=test-table [id=97991b3b060744ec8dc2e132401a94e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:35.815236 18975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41cec91ece94469a9405e88dc9f8d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:35.827883 19356 tablet_bootstrap.cc:492] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4: Bootstrap starting.
I20260812 02:42:35.830328 19358 tablet_bootstrap.cc:492] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141: Bootstrap starting.
I20260812 02:42:35.832412 19357 tablet_bootstrap.cc:492] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9: Bootstrap starting.
I20260812 02:42:35.833784 19356 tablet_bootstrap.cc:654] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:35.833937 19358 tablet_bootstrap.cc:654] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:35.835299 19358 log.cc:824] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:35.835296 19356 log.cc:824] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:35.836166 19357 tablet_bootstrap.cc:654] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:35.837630 19357 log.cc:824] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:35.838939 19358 tablet_bootstrap.cc:492] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141: No bootstrap required, opened a new log
I20260812 02:42:35.839258 19358 ts_tablet_manager.cc:1397] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 02:42:35.840209 19356 tablet_bootstrap.cc:492] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4: No bootstrap required, opened a new log
I20260812 02:42:35.840553 19356 ts_tablet_manager.cc:1397] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260812 02:42:35.840817 19357 tablet_bootstrap.cc:492] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9: No bootstrap required, opened a new log
I20260812 02:42:35.841089 19357 ts_tablet_manager.cc:1397] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 02:42:35.849265 19357 raft_consensus.cc:348] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:35.849820 19357 raft_consensus.cc:374] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:35.849948 19357 raft_consensus.cc:729] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb7836ec0a041fb804644686f9633f9, State: Initialized, Role: FOLLOWER
I20260812 02:42:35.850706 19357 consensus_queue.cc:260] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:35.850646 19358 raft_consensus.cc:348] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:35.851109 19358 raft_consensus.cc:374] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:35.851199 19358 raft_consensus.cc:729] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e13a4dfa42a843dc8429345d27313141, State: Initialized, Role: FOLLOWER
I20260812 02:42:35.851940 19358 consensus_queue.cc:260] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:35.852460 19356 raft_consensus.cc:348] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:35.853024 19356 raft_consensus.cc:374] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:35.853173 19356 raft_consensus.cc:729] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3dcb5614404ee89647827f758d94a4, State: Initialized, Role: FOLLOWER
I20260812 02:42:35.853904 19189 heartbeater.cc:503] Master 127.17.196.254:37767 was elected leader, sending a full tablet report...
I20260812 02:42:35.853935 19356 consensus_queue.cc:260] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:35.854940 19357 ts_tablet_manager.cc:1428] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260812 02:42:35.856170 19335 heartbeater.cc:503] Master 127.17.196.254:37767 was elected leader, sending a full tablet report...
I20260812 02:42:35.857146 19358 ts_tablet_manager.cc:1428] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
I20260812 02:42:35.859000 19356 ts_tablet_manager.cc:1428] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4: Time spent starting tablet: real 0.018s user 0.011s sys 0.005s
W20260812 02:42:35.902169 19190 tablet.cc:2369] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:42:35.949872 19336 tablet.cc:2369] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:42:36.048803 19366 raft_consensus.cc:482] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:36.049188 19366 raft_consensus.cc:504] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:36.051198 19366 leader_election.cc:288] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e13a4dfa42a843dc8429345d27313141 (127.17.196.195:44167), 6fb7836ec0a041fb804644686f9633f9 (127.17.196.194:32899)
I20260812 02:42:36.059746 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.051710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:38769 (local address 127.17.196.195:44167)
0812 02:42:36.052181 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:36.052187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:36.052466 (+ 279us) server_negotiation.cc:407] Connection header received
0812 02:42:36.052741 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:36.052745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:36.052805 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:36.052909 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:36.053442 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.054509 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.057235 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.058618 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.058809 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.058893 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:36.059248 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 02:42:36.059392 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316}
I20260812 02:42:36.059791 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.051936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:52309 (local address 127.17.196.194:32899)
0812 02:42:36.052179 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:36.052187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:36.052883 (+ 696us) server_negotiation.cc:407] Connection header received
0812 02:42:36.053124 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:36.053128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:36.053188 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:36.053292 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:36.054048 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.054978 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.057455 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.058863 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.059015 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.059065 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:36.059418 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 02:42:36.059554 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:42:36.059791 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.051506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:44167 (local address 127.17.196.193:38769)
0812 02:42:36.052342 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:36.052359 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:36.052564 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:36.052940 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:36.052945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:36.052973 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:36.053294 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:36.053303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.054712 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.054716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:36.057057 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:36.057073 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.058848 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.058853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:36.059061 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.059104 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 02:42:36.059208 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 02:42:36.059357 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":504,"threads_started":1}
I20260812 02:42:36.060088 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.051710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:32899 (local address 127.17.196.193:52309)
0812 02:42:36.052751 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:36.052766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:36.052939 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:36.053312 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:36.053316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:36.053343 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:36.053825 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:36.053841 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.055168 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.055175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:36.057290 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:36.057301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.059090 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.059096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:36.059262 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.059290 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 02:42:36.059370 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 02:42:36.059501 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":1,"thread_start_us":690,"threads_started":1}
I20260812 02:42:36.061125 19288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41cec91ece94469a9405e88dc9f8d81" candidate_uuid: "ce3dcb5614404ee89647827f758d94a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e13a4dfa42a843dc8429345d27313141" is_pre_election: true
I20260812 02:42:36.061208 19141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41cec91ece94469a9405e88dc9f8d81" candidate_uuid: "ce3dcb5614404ee89647827f758d94a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb7836ec0a041fb804644686f9633f9" is_pre_election: true
I20260812 02:42:36.061707 19288 raft_consensus.cc:2454] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce3dcb5614404ee89647827f758d94a4 in term 0.
I20260812 02:42:36.061836 19141 raft_consensus.cc:2454] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce3dcb5614404ee89647827f758d94a4 in term 0.
I20260812 02:42:36.062436 18926 leader_election.cc:302] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3dcb5614404ee89647827f758d94a4, e13a4dfa42a843dc8429345d27313141; no voters:
I20260812 02:42:36.062862 19366 raft_consensus.cc:2781] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:36.063019 19366 raft_consensus.cc:482] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:36.063161 19366 raft_consensus.cc:3037] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:36.065429 19366 raft_consensus.cc:504] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:36.066195 19366 leader_election.cc:288] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [CANDIDATE]: Term 1 election: Requested vote from peers e13a4dfa42a843dc8429345d27313141 (127.17.196.195:44167), 6fb7836ec0a041fb804644686f9633f9 (127.17.196.194:32899)
I20260812 02:42:36.066732 19288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41cec91ece94469a9405e88dc9f8d81" candidate_uuid: "ce3dcb5614404ee89647827f758d94a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e13a4dfa42a843dc8429345d27313141"
I20260812 02:42:36.066836 19141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41cec91ece94469a9405e88dc9f8d81" candidate_uuid: "ce3dcb5614404ee89647827f758d94a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb7836ec0a041fb804644686f9633f9"
I20260812 02:42:36.066953 19288 raft_consensus.cc:3037] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:36.067111 19141 raft_consensus.cc:3037] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:36.069281 19288 raft_consensus.cc:2454] T e41cec91ece94469a9405e88dc9f8d81 P e13a4dfa42a843dc8429345d27313141 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce3dcb5614404ee89647827f758d94a4 in term 1.
I20260812 02:42:36.069365 19141 raft_consensus.cc:2454] T e41cec91ece94469a9405e88dc9f8d81 P 6fb7836ec0a041fb804644686f9633f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce3dcb5614404ee89647827f758d94a4 in term 1.
I20260812 02:42:36.069828 18926 leader_election.cc:302] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3dcb5614404ee89647827f758d94a4, e13a4dfa42a843dc8429345d27313141; no voters:
I20260812 02:42:36.070195 19366 raft_consensus.cc:2781] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:36.070729 19366 raft_consensus.cc:686] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [term 1 LEADER]: Becoming Leader. State: Replica: ce3dcb5614404ee89647827f758d94a4, State: Running, Role: LEADER
I20260812 02:42:36.071234 19366 consensus_queue.cc:237] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } }
I20260812 02:42:36.076366 18835 catalog_manager.cc:5495] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce3dcb5614404ee89647827f758d94a4 (127.17.196.193). New cstate: current_term: 1 leader_uuid: "ce3dcb5614404ee89647827f758d94a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e13a4dfa42a843dc8429345d27313141" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 44167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce3dcb5614404ee89647827f758d94a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fb7836ec0a041fb804644686f9633f9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 32899 } health_report { overall_health: UNKNOWN } } }
W20260812 02:42:36.095643 19045 tablet.cc:2369] T e41cec91ece94469a9405e88dc9f8d81 P ce3dcb5614404ee89647827f758d94a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:42:36.118311 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.102237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47582 (local address 127.17.196.193:35109)
0812 02:42:36.102452 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:36.102457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:36.103178 (+ 721us) server_negotiation.cc:407] Connection header received
0812 02:42:36.103420 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:36.103432 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:36.103503 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:36.103623 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:36.105196 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.106018 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.107311 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.107725 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.107861 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.112265 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:36.112282 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:36.112298 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:36.112338 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:36.115309 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.116027 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.116035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.116037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.116209 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.116610 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.116615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.116618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.117026 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:36.117341 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:36.117683 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 02:42:36.117921 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[18787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502556, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502556, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:42:36.175499 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.158538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47598 (local address 127.17.196.193:35109)
0812 02:42:36.158754 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:36.158762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:36.159178 (+ 416us) server_negotiation.cc:407] Connection header received
0812 02:42:36.159337 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:36.159343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:36.159447 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:36.159591 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:36.161029 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.161913 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.163582 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.164054 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.164172 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.168405 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:36.168412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:36.168417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:36.168556 (+ 139us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:36.172410 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.173357 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.173362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.173365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.173592 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.173953 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.173956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.173959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.174350 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:36.174693 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:36.174897 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 02:42:36.175087 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[18787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:42:36.232928 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.216193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47608 (local address 127.17.196.193:35109)
0812 02:42:36.216488 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:36.216495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:36.216742 (+ 247us) server_negotiation.cc:407] Connection header received
0812 02:42:36.216976 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:36.216984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:36.217092 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:36.217230 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:36.218993 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.219874 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.221300 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.221889 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.222043 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.226990 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:36.226996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:36.227000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:36.227049 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:36.230243 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.230973 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.230977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.230980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.231132 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.231552 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.231556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.231559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.231873 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:36.232103 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:36.232400 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 02:42:36.232654 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
W20260812 02:42:36.235329 18954 tablet_service.cc:2908] Rejecting scan request for tablet e41cec91ece94469a9405e88dc9f8d81: Uninitialized: clean time has not yet been initialized
I20260812 02:42:36.238839 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 18905
I20260812 02:42:36.253785 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.239895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.17.196.193:35109)
0812 02:42:36.240095 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:36.240099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:36.240597 (+ 498us) server_negotiation.cc:407] Connection header received
0812 02:42:36.240780 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:36.240784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:36.240843 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:36.240939 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:36.242559 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.243355 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.244722 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.245194 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.245342 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.247801 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:36.247806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:36.247810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:36.247844 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:36.250859 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.251667 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.251674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.251678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.251815 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.252204 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.252212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.252215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.252536 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:36.252800 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:36.253126 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 02:42:36.253395 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:42:36.254616 19039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:36.351945 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 18905
I20260812 02:42:36.377679 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19050
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:42:36.395970 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.378657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.17.196.194:32899)
0812 02:42:36.378871 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:36.378876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:36.378984 (+ 108us) server_negotiation.cc:407] Connection header received
0812 02:42:36.379201 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:36.379206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:36.379278 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:36.379368 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:36.380683 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.381442 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.383094 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.383500 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.383653 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.388571 (+ 4918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:36.388590 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:36.388609 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:36.388657 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:36.392493 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.393297 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.393306 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.393309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.393588 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.394251 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.394256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.394258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.394691 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:36.394916 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:36.395302 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 02:42:36.395485 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:42:36.396776 19184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:36.490425 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 19050
I20260812 02:42:36.512405 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19195
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[18787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:42:36.529970 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:36.513285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.17.196.195:44167)
0812 02:42:36.513453 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:36.513458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:36.513492 (+ 34us) server_negotiation.cc:407] Connection header received
0812 02:42:36.513686 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:36.513690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:36.513761 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:36.513852 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:36.515319 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.516048 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.517681 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:36.518142 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:36.518263 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:36.523255 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:36.523273 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:36.523292 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:36.523336 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:36.526822 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.527662 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.527679 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.527682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.527841 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:36.528276 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:36.528280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:36.528282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:36.528695 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:36.528919 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:36.529327 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 02:42:36.529688 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 02:42:36.530700 19330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:36.621784 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 19195
I20260812 02:42:36.642894 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 18804
I20260812 02:42:36.644320 18873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:36.756039 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 18804
2026-08-12T02:42:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4729 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[19391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[19391](info): set up 2 sockets
Aug 12 02:42:36 dist-test-slave-ss4n krb5kdc[19391](info): commencing operation
krb5kdc: starting...
W20260812 02:42:38.017171 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:42:38 dist-test-slave-ss4n krb5kdc[19391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:42:38.251173 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42451
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:42451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:38.591923 19404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:38.592263 19404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:38.592360 19404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:38.599709 19404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:38.599834 19404 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 02:42:38.599879 19404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:38.599934 19404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:38.599992 19404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:38.600056 19404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:38.610291 19404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:42451
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:38.611665 19404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:38.612993 19404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:38.626667 19409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:38.626641 19410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:38.626793 19412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:38.627473 19404 server_base.cc:1034] running on GCE node
I20260812 02:42:38.628325 19404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:38.629980 19404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:38.631249 19404 hybrid_clock.cc:648] HybridClock initialized: now 1786502558631177 us; error 35 us; skew 500 ppm
I20260812 02:42:38.631673 19404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:38 dist-test-slave-ss4n krb5kdc[19391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502558, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:38.635456 19404 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:42:38.637306 19404 webserver.cc:460] Webserver started at http://127.17.196.254:34795/ using document root <none> and password file <none>
I20260812 02:42:38.638187 19404 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:38.638290 19404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:38.638713 19404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:38.642163 19404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "2d31bd36dee34d0a9d7c20ad11361a1e"
format_stamp: "Formatted at 2026-08-12 02:42:38 on dist-test-slave-ss4n"
server_key: "9b3f28a7420c7762d34d114dba58706e"
server_key_iv: "5051875345e44ad24ff3256f30b73b77"
server_key_version: "encryptionkey@0"
I20260812 02:42:38.643059 19404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "2d31bd36dee34d0a9d7c20ad11361a1e"
format_stamp: "Formatted at 2026-08-12 02:42:38 on dist-test-slave-ss4n"
server_key: "9b3f28a7420c7762d34d114dba58706e"
server_key_iv: "5051875345e44ad24ff3256f30b73b77"
server_key_version: "encryptionkey@0"
I20260812 02:42:38.649130 19404 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 02:42:38.653266 19419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:38.654980 19404 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:42:38.655205 19404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "2d31bd36dee34d0a9d7c20ad11361a1e"
format_stamp: "Formatted at 2026-08-12 02:42:38 on dist-test-slave-ss4n"
server_key: "9b3f28a7420c7762d34d114dba58706e"
server_key_iv: "5051875345e44ad24ff3256f30b73b77"
server_key_version: "encryptionkey@0"
I20260812 02:42:38.655462 19404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:38.671679 19404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:38.677095 19404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:38.677430 19404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:38.703408 19404 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:42451
I20260812 02:42:38.703433 19483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:42451 every 8 connection(s)
I20260812 02:42:38.704937 19404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:38.709582 19484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:38.713801 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 19404
I20260812 02:42:38.714076 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:42:38.714763 18195 external_mini_cluster.cc:1428] Setting key b115028d68265d48f9673b6790725a44
I20260812 02:42:38.725673 19484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e: Bootstrap starting.
I20260812 02:42:38.729930 19484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:38.731140 19484 log.cc:824] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:42:38 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:38.735013 19484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e: No bootstrap required, opened a new log
I20260812 02:42:38.742702 19487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:38.717202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.17.196.254:42451)
0812 02:42:38.718190 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:38.718208 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:38.718264 (+ 56us) server_negotiation.cc:407] Connection header received
0812 02:42:38.720580 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:38.720627 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:38.721301 (+ 674us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:38.721879 (+ 578us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:38.723538 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:38.725782 (+ 2244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:38.727178 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:38.728439 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:38.728666 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:38.733628 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:38.733639 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:38.733660 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:38.733708 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:38.737381 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:38.738603 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:38.738612 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:38.738621 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:38.738828 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:38.739329 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:38.739333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:38.739335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:38.740058 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:38.740329 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:38.740903 (+ 574us) server_negotiation.cc:299] Negotiation successful
0812 02:42:38.741382 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":435,"threads_started":1}
I20260812 02:42:38.745443 19484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d31bd36dee34d0a9d7c20ad11361a1e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42451 } }
I20260812 02:42:38.746017 19484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:38.746160 19484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d31bd36dee34d0a9d7c20ad11361a1e, State: Initialized, Role: FOLLOWER
I20260812 02:42:38.746752 19484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d31bd36dee34d0a9d7c20ad11361a1e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42451 } }
I20260812 02:42:38.746990 19484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:38.747103 19484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:38.747287 19484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:38.750195 19484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d31bd36dee34d0a9d7c20ad11361a1e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42451 } }
I20260812 02:42:38.750747 19484 leader_election.cc:302] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d31bd36dee34d0a9d7c20ad11361a1e; no voters:
I20260812 02:42:38.751443 19484 leader_election.cc:288] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:38.751629 19489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:38.753080 19489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [term 1 LEADER]: Becoming Leader. State: Replica: 2d31bd36dee34d0a9d7c20ad11361a1e, State: Running, Role: LEADER
I20260812 02:42:38.753793 19489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d31bd36dee34d0a9d7c20ad11361a1e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42451 } }
I20260812 02:42:38.754189 19484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:38.757210 19491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d31bd36dee34d0a9d7c20ad11361a1e. Latest consensus state: current_term: 1 leader_uuid: "2d31bd36dee34d0a9d7c20ad11361a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d31bd36dee34d0a9d7c20ad11361a1e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42451 } } }
I20260812 02:42:38.757768 19491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:38.757784 19490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d31bd36dee34d0a9d7c20ad11361a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d31bd36dee34d0a9d7c20ad11361a1e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42451 } } }
I20260812 02:42:38.758834 19490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:38.767187 19495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:38.774044 19495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:38.785008 19495 catalog_manager.cc:1347] Generated new cluster ID: cbe28c79e44a4cc28625533e95cbe852
I20260812 02:42:38.785159 19495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:38.816382 19495 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:38.817937 19495 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:38.828619 19495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e: Generated new TSK 0
I20260812 02:42:38.829684 19495 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:42:38.890352 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42451
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:39.259599 19511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:39.259985 19511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:39.260185 19511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:39.268975 19511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:39.269120 19511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:39.269299 19511 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:42:39.269426 19511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:42:39.281468 19511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:42451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:39.283186 19511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:39.284693 19511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:39.299690 19516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:39.299734 19520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:39.299800 19517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:39.301251 19511 server_base.cc:1034] running on GCE node
I20260812 02:42:39.302155 19511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:39.303315 19511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:39.304610 19511 hybrid_clock.cc:648] HybridClock initialized: now 1786502559304546 us; error 41 us; skew 500 ppm
I20260812 02:42:39.304989 19511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:39 dist-test-slave-ss4n krb5kdc[19391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502559, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:39.309023 19511 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:42:39.310899 19511 webserver.cc:460] Webserver started at http://127.17.196.193:38343/ using document root <none> and password file <none>
I20260812 02:42:39.311676 19511 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:39.311825 19511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:39.312191 19511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:39.315476 19511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "74730d68fe99401aa2633bd96391bf31"
format_stamp: "Formatted at 2026-08-12 02:42:39 on dist-test-slave-ss4n"
server_key: "0e8b99435ded1e67572cd07824f16992"
server_key_iv: "dfd28f2b8a5abe6ec1897886606158c1"
server_key_version: "encryptionkey@0"
I20260812 02:42:39.316325 19511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "74730d68fe99401aa2633bd96391bf31"
format_stamp: "Formatted at 2026-08-12 02:42:39 on dist-test-slave-ss4n"
server_key: "0e8b99435ded1e67572cd07824f16992"
server_key_iv: "dfd28f2b8a5abe6ec1897886606158c1"
server_key_version: "encryptionkey@0"
I20260812 02:42:39.322180 19511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:42:39.326414 19528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:39.328003 19511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:42:39.328243 19511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "74730d68fe99401aa2633bd96391bf31"
format_stamp: "Formatted at 2026-08-12 02:42:39 on dist-test-slave-ss4n"
server_key: "0e8b99435ded1e67572cd07824f16992"
server_key_iv: "dfd28f2b8a5abe6ec1897886606158c1"
server_key_version: "encryptionkey@0"
I20260812 02:42:39.328583 19511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:39.367892 19511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:39.373313 19511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:39.373690 19511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:39.377015 19511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:39.377151 19511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:39.377316 19511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:39.377403 19511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:42:39 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502559, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:39.404906 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:39.378101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:34297 (local address 127.17.196.254:42451)
0812 02:42:39.378632 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:39.378637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:39.380163 (+ 1526us) server_negotiation.cc:407] Connection header received
0812 02:42:39.382415 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:39.382421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:39.382541 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:39.382674 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:39.385128 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.386145 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.387833 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.388387 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.388559 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:39.395278 (+ 6719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:39.395288 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:39.395296 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:39.395362 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:39.400367 (+ 5005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:39.401670 (+ 1303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:39.401680 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:39.401687 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:39.401913 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:39.402797 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:39.402806 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:39.402811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:39.403207 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:39.403383 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:39.404255 (+ 872us) server_negotiation.cc:299] Negotiation successful
0812 02:42:39.404521 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":245,"threads_started":1}
I20260812 02:42:39.406368 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:39.379031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42451 (local address 127.17.196.193:34297)
0812 02:42:39.379953 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:39.380010 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:39.382080 (+ 2070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:39.383006 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:39.383021 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:39.383877 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:39.384923 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:39.384946 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.386331 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.386337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:39.387677 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:39.387690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.388724 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.388733 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:39.389210 (+ 477us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:39.390344 (+ 1134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:39.390397 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:39.394835 (+ 4438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:39.400606 (+ 5771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:39.400625 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:39.400652 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:39.401374 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:39.402127 (+ 753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:39.402149 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:39.402152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:39.402603 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:39.403369 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:39.403378 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:39.403815 (+ 437us) client_negotiation.cc:769] Sending connection context
0812 02:42:39.404127 (+ 312us) client_negotiation.cc:240] Negotiation successful
0812 02:42:39.404736 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":435,"threads_started":1}
I20260812 02:42:39.430958 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:39.413083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:53219 (local address 127.17.196.254:42451)
0812 02:42:39.413253 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:39.413259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:39.414313 (+ 1054us) server_negotiation.cc:407] Connection header received
0812 02:42:39.414525 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:39.414533 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:39.414701 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:39.414816 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:39.416627 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.417619 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.419383 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.420405 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.420603 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:39.423985 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:39.423994 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:39.423999 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:39.424060 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:39.427444 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:39.428641 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:39.428645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:39.428647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:39.428783 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:39.429294 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:39.429297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:39.429300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:39.429663 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:39.429784 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:39.430497 (+ 713us) server_negotiation.cc:299] Negotiation successful
0812 02:42:39.430705 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 02:42:39.431731 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:39.413203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42451 (local address 127.17.196.193:53219)
0812 02:42:39.414199 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:39.414216 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:39.414390 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:39.414902 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:39.414911 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:39.415761 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:39.416404 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:39.416425 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.417876 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.417884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:39.419204 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:39.419219 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.420749 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.420755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:39.421040 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:39.421897 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:39.421930 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:39.423688 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:39.427620 (+ 3932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:39.427630 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:39.427637 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:39.428477 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:39.428915 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:39.428924 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:39.428932 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:39.429168 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:39.429799 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:39.429808 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:39.430128 (+ 320us) client_negotiation.cc:769] Sending connection context
0812 02:42:39.430377 (+ 249us) client_negotiation.cc:240] Negotiation successful
0812 02:42:39.430915 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":533,"threads_started":1}
I20260812 02:42:39.446020 19511 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:35479
I20260812 02:42:39.446043 19664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:35479 every 8 connection(s)
I20260812 02:42:39.447419 19511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:42:39.450047 19665 heartbeater.cc:347] Connected to a master server at 127.17.196.254:42451
I20260812 02:42:39.450338 19665 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:39.451102 19665 heartbeater.cc:511] Master 127.17.196.254:42451 requested a full tablet report, sending...
I20260812 02:42:39.453662 19436 ts_manager.cc:194] Registered new tserver with Master: 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479)
I20260812 02:42:39.455536 19436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:34297
I20260812 02:42:39.457971 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 19511
I20260812 02:42:39.458227 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:42:39.458957 18195 external_mini_cluster.cc:1428] Setting key 24a1b36977c7344d7d06fa520edb43b8
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:42:39.514657 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42451
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:39.865983 19670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:39.866343 19670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:39.866556 19670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:39.874130 19670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:39.874315 19670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:39.874486 19670 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:42:39.874639 19670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:42:39.884805 19670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:42451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:39.886286 19670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:39.887805 19670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:39.902390 19679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:39.902424 19676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:39.902495 19675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:39.903437 19670 server_base.cc:1034] running on GCE node
I20260812 02:42:39.904161 19670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:39.905203 19670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:39.906639 19670 hybrid_clock.cc:648] HybridClock initialized: now 1786502559906422 us; error 58 us; skew 500 ppm
I20260812 02:42:39.907135 19670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:39 dist-test-slave-ss4n krb5kdc[19391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502559, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:39.911198 19670 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:42:39.912884 19670 webserver.cc:460] Webserver started at http://127.17.196.194:39693/ using document root <none> and password file <none>
I20260812 02:42:39.913843 19670 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:39.913995 19670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:39.914348 19670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:39.919009 19670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "45e7ed64d69f44efa05d46b34a718412"
format_stamp: "Formatted at 2026-08-12 02:42:39 on dist-test-slave-ss4n"
server_key: "0e10e06796733a2354c2801305cb733a"
server_key_iv: "d0f10c5a81aefb70d4b8a546f5771c07"
server_key_version: "encryptionkey@0"
I20260812 02:42:39.920215 19670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "45e7ed64d69f44efa05d46b34a718412"
format_stamp: "Formatted at 2026-08-12 02:42:39 on dist-test-slave-ss4n"
server_key: "0e10e06796733a2354c2801305cb733a"
server_key_iv: "d0f10c5a81aefb70d4b8a546f5771c07"
server_key_version: "encryptionkey@0"
I20260812 02:42:39.927843 19670 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260812 02:42:39.932163 19686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:39.934238 19670 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:42:39.934432 19670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "45e7ed64d69f44efa05d46b34a718412"
format_stamp: "Formatted at 2026-08-12 02:42:39 on dist-test-slave-ss4n"
server_key: "0e10e06796733a2354c2801305cb733a"
server_key_iv: "d0f10c5a81aefb70d4b8a546f5771c07"
server_key_version: "encryptionkey@0"
I20260812 02:42:39.934681 19670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:39.955281 19670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:39.960135 19670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:39.960441 19670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:39.963697 19670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:39.963785 19670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:39.963936 19670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:39.964005 19670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:42:39 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502559, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:39.994786 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:39.964778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:54183 (local address 127.17.196.254:42451)
0812 02:42:39.965316 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:39.965323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:39.966883 (+ 1560us) server_negotiation.cc:407] Connection header received
0812 02:42:39.970459 (+ 3576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:39.970470 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:39.970588 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:39.970756 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:39.973528 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.974601 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.978149 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.978881 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.979056 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:39.985235 (+ 6179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:39.985244 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:39.985255 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:39.985332 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:39.989769 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:39.991194 (+ 1425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:39.991203 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:39.991206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:39.991395 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:39.992066 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:39.992072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:39.992076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:39.992485 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:39.992729 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:39.993922 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0812 02:42:39.994242 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":270,"threads_started":1}
I20260812 02:42:39.996834 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:39.965640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42451 (local address 127.17.196.194:54183)
0812 02:42:39.966671 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:39.966735 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:39.969998 (+ 3263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:39.971129 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:39.971149 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:39.972133 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:39.973248 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:39.973294 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.974814 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.974822 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:39.977832 (+ 3010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:39.977855 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:39.979476 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:39.979489 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:39.979855 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:39.980805 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:39.980858 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:39.984896 (+ 4038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:39.990002 (+ 5106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:39.990016 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:39.990061 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:39.990963 (+ 902us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:39.991566 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:39.991573 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:39.991577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:39.991884 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:39.992668 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:39.992680 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:39.993328 (+ 648us) client_negotiation.cc:769] Sending connection context
0812 02:42:39.993748 (+ 420us) client_negotiation.cc:240] Negotiation successful
0812 02:42:39.994527 (+ 779us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":434,"threads_started":1}
I20260812 02:42:40.013116 19670 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:35317
I20260812 02:42:40.013131 19815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:35317 every 8 connection(s)
I20260812 02:42:40.014925 19670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:42:40.017789 19816 heartbeater.cc:347] Connected to a master server at 127.17.196.254:42451
I20260812 02:42:40.018116 19816 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:40.018993 19816 heartbeater.cc:511] Master 127.17.196.254:42451 requested a full tablet report, sending...
I20260812 02:42:40.019188 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 19670
I20260812 02:42:40.019412 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:42:40.020210 18195 external_mini_cluster.cc:1428] Setting key 243aca4dbc5910097ee8aa392fe15910
I20260812 02:42:40.020681 19436 ts_manager.cc:194] Registered new tserver with Master: 45e7ed64d69f44efa05d46b34a718412 (127.17.196.194:35317)
I20260812 02:42:40.022032 19436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:54183
I20260812 02:42:40.023779 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.005314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:45539 (local address 127.17.196.254:42451)
0812 02:42:40.005614 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.005625 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.006270 (+ 645us) server_negotiation.cc:407] Connection header received
0812 02:42:40.006583 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.006590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.006699 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.006801 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:40.008623 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.009979 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.012433 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.013185 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.013321 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.016117 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:40.016126 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:40.016133 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:40.016179 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:40.020318 (+ 4139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:40.021312 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:40.021318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:40.021322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:40.021536 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:40.021924 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:40.021931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:40.021936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:40.022471 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:40.022620 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.023182 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.023420 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:42:40.023932 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.005134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42451 (local address 127.17.196.194:45539)
0812 02:42:40.006122 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:40.006148 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:40.006364 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:40.006874 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:40.006882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:40.007769 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:40.008381 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.008407 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.010223 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.010231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.012188 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.012217 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.013621 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.013626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.013794 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.014522 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:40.014546 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:40.015806 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:40.020518 (+ 4712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:40.020524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:40.020528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:40.021185 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:40.021660 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:40.021665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:40.021668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:40.021808 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:40.022715 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:40.022722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:40.022913 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 02:42:40.023073 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 02:42:40.023447 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":563,"threads_started":1}
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:42:40.078047 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42451
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:40.423343 19821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:40.423720 19821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:40.423827 19821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:40.431631 19821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:40.431752 19821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:40.431840 19821 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:42:40.431922 19821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:42:40.442453 19821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:42451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:40.443792 19821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:40.445348 19821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:42:40.459142 19665 heartbeater.cc:503] Master 127.17.196.254:42451 was elected leader, sending a full tablet report...
W20260812 02:42:40.461791 19826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:40.461870 19827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:40.462364 19829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:40.462328 19821 server_base.cc:1034] running on GCE node
I20260812 02:42:40.463295 19821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:40.464784 19821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:40.465979 19821 hybrid_clock.cc:648] HybridClock initialized: now 1786502560465905 us; error 60 us; skew 500 ppm
I20260812 02:42:40.466441 19821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:40 dist-test-slave-ss4n krb5kdc[19391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502560, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:40.477126 19821 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:42:40.478974 19821 webserver.cc:460] Webserver started at http://127.17.196.195:46387/ using document root <none> and password file <none>
I20260812 02:42:40.479717 19821 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:40.479815 19821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:40.480198 19821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:40.483402 19821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "0c4c2bc2fb5745cfb066cb5d25554260"
format_stamp: "Formatted at 2026-08-12 02:42:40 on dist-test-slave-ss4n"
server_key: "c242510dd4158ad9e6c204da6c7664ca"
server_key_iv: "5fe094c837213f6369a48eecdd46722f"
server_key_version: "encryptionkey@0"
I20260812 02:42:40.484200 19821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "0c4c2bc2fb5745cfb066cb5d25554260"
format_stamp: "Formatted at 2026-08-12 02:42:40 on dist-test-slave-ss4n"
server_key: "c242510dd4158ad9e6c204da6c7664ca"
server_key_iv: "5fe094c837213f6369a48eecdd46722f"
server_key_version: "encryptionkey@0"
I20260812 02:42:40.489840 19821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:42:40.493646 19836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:40.495059 19821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:42:40.495249 19821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "0c4c2bc2fb5745cfb066cb5d25554260"
format_stamp: "Formatted at 2026-08-12 02:42:40 on dist-test-slave-ss4n"
server_key: "c242510dd4158ad9e6c204da6c7664ca"
server_key_iv: "5fe094c837213f6369a48eecdd46722f"
server_key_version: "encryptionkey@0"
I20260812 02:42:40.495465 19821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:40.516333 19821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:40.521588 19821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:40.521867 19821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:40.524930 19821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:40.525030 19821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:40.525233 19821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:40.525332 19821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:42:40 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502560, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:40.551630 19846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.526054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:40653 (local address 127.17.196.254:42451)
0812 02:42:40.526560 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.526569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.528393 (+ 1824us) server_negotiation.cc:407] Connection header received
0812 02:42:40.530597 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.530604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.530707 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.530839 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:40.532550 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.534094 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.536957 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.537446 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.537661 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.543020 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:40.543031 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:40.543047 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:40.543107 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:40.547221 (+ 4114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:40.548598 (+ 1377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:40.548602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:40.548605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:40.548839 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:40.549471 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:40.549520 (+ 49us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:40.549528 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:40.549837 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:40.550020 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.550895 (+ 875us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.551170 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":233,"threads_started":1}
I20260812 02:42:40.552959 19847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.526807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42451 (local address 127.17.196.195:40653)
0812 02:42:40.527831 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:40.527882 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:40.530270 (+ 2388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:40.531106 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:40.531115 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:40.531661 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:40.532293 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.532312 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.534291 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.534300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.536776 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.536794 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.537781 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.537795 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.538098 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.539102 (+ 1004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:40.539147 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:40.542724 (+ 3577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:40.547505 (+ 4781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:40.547519 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:40.547540 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:40.548368 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:40.549035 (+ 667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:40.549048 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:40.549052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:40.549314 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:40.550003 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:40.550012 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:40.550443 (+ 431us) client_negotiation.cc:769] Sending connection context
0812 02:42:40.550763 (+ 320us) client_negotiation.cc:240] Negotiation successful
0812 02:42:40.551419 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":461,"threads_started":1}
I20260812 02:42:40.584084 19846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.565049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:56675 (local address 127.17.196.254:42451)
0812 02:42:40.565297 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.565307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.566017 (+ 710us) server_negotiation.cc:407] Connection header received
0812 02:42:40.566373 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.566379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.566534 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.566659 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:40.568400 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.569234 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.572325 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.572963 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.573156 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.576462 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:40.576468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:40.576471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:40.576508 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:40.580402 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:40.581652 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:40.581657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:40.581660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:40.581804 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:40.582491 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:40.582497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:40.582501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:40.582830 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:40.582964 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.583605 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.583799 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:42:40.584801 19922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.564935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42451 (local address 127.17.196.195:56675)
0812 02:42:40.565842 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:40.565871 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:40.566190 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:40.566734 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:40.566743 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:40.567658 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:40.568203 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.568222 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.569398 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.569407 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.572078 (+ 2671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.572113 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.573255 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.573268 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.573602 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.574482 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:40.574521 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:40.576174 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:40.580566 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:40.580572 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:40.580576 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:40.581383 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:40.581967 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:40.581980 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:40.581984 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:40.582311 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:40.582982 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:40.582992 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:40.583294 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 02:42:40.583496 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 02:42:40.584045 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":422,"threads_started":1}
I20260812 02:42:40.592941 19821 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:36365
I20260812 02:42:40.592974 19968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:36365 every 8 connection(s)
I20260812 02:42:40.594251 19821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:42:40.596601 19969 heartbeater.cc:347] Connected to a master server at 127.17.196.254:42451
I20260812 02:42:40.596873 19969 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:40.597689 19969 heartbeater.cc:511] Master 127.17.196.254:42451 requested a full tablet report, sending...
I20260812 02:42:40.599225 19435 ts_manager.cc:194] Registered new tserver with Master: 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365)
I20260812 02:42:40.600389 19435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:40653
I20260812 02:42:40.602883 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 19821
I20260812 02:42:40.603094 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:42:40.603570 18195 external_mini_cluster.cc:1428] Setting key e8687b27fe3fa0f3cce82ef0465c4ee0
I20260812 02:42:40.607004 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:42:40 dist-test-slave-ss4n krb5kdc[19391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:42:40 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:40.664029 19846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.648022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.17.196.254:42451)
0812 02:42:40.648246 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.648253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.648572 (+ 319us) server_negotiation.cc:407] Connection header received
0812 02:42:40.648693 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.648698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.648791 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.648896 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:40.650733 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.651554 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.653026 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.653587 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.653698 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.658099 (+ 4401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:40.658104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:40.658108 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:40.658149 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:40.661382 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:40.662152 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:40.662157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:40.662160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:40.662312 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:40.662743 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:40.662747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:40.662749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:40.663040 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:40.663281 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.663545 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.663737 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 02:42:40.669054 19435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:42:40.676285 19435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:40.701850 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.689527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35317 (local address 127.0.0.1:42648)
0812 02:42:40.690712 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:40.690777 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:40.691013 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:40.692451 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:40.692460 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:40.692503 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:40.693417 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.693462 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.695344 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.695350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.697983 (+ 2633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.698000 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.700663 (+ 2663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.700671 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.700955 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.701027 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 02:42:40.701149 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 02:42:40.701371 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"spinlock_wait_cycles":305920,"thread_start_us":993,"threads_started":1}
I20260812 02:42:40.702010 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.690292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.17.196.193:35479)
0812 02:42:40.692000 (+ 1708us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.692009 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.692638 (+ 629us) server_negotiation.cc:407] Connection header received
0812 02:42:40.693867 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.693875 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.694143 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.694312 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:40.695159 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.696823 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.699061 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.700494 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.700679 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.700867 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.701466 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.701696 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1545,"thread_start_us":1523,"threads_started":1}
I20260812 02:42:40.702270 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.689694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.17.196.194:35317)
0812 02:42:40.691670 (+ 1976us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.691682 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.691717 (+ 35us) server_negotiation.cc:407] Connection header received
0812 02:42:40.691831 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.691839 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.692156 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.692633 (+ 477us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:40.693657 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.695190 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.698182 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.699736 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.701452 (+ 1716us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.701676 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.701780 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.701957 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1801,"thread_start_us":1668,"threads_started":1}
I20260812 02:42:40.703853 19985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.689791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:35479 (local address 127.0.0.1:34094)
0812 02:42:40.692514 (+ 2723us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:40.692541 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:40.693674 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:40.694609 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:40.694614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:40.694641 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:40.694981 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.694992 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.697014 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.697020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.698845 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.698858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.701036 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.701043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.701222 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.701261 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:42:40.702413 (+ 1152us) client_negotiation.cc:240] Negotiation successful
0812 02:42:40.702576 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2561,"thread_start_us":2677,"threads_started":1}
I20260812 02:42:40.705278 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.691676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:36365 (local address 127.0.0.1:53804)
0812 02:42:40.695404 (+ 3728us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:40.695419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:40.695585 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:40.697309 (+ 1724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:40.697314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:40.697347 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:40.697912 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.697922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.700172 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.700178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.702454 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.702469 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.704429 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.704440 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.704639 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.704678 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:42:40.704765 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 02:42:40.704898 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3573,"mutex_wait_us":83,"thread_start_us":1894,"threads_started":1}
I20260812 02:42:40.706044 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.692098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.17.196.195:36365)
0812 02:42:40.693341 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.693352 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.695539 (+ 2187us) server_negotiation.cc:407] Connection header received
0812 02:42:40.695773 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.695780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.696016 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.696665 (+ 649us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:40.698141 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.699827 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.702661 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.703870 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.704086 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.704265 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.705549 (+ 1284us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.705719 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":1590,"threads_started":1}
I20260812 02:42:40.706764 19741 tablet_service.cc:1467] Processing CreateTablet for tablet ea0c6538a3c84b02b8a691d56694e69f (DEFAULT_TABLE table=test-table [id=cd71204ea6a54516b7c6eca1968b94e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:40.708074 19741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0c6538a3c84b02b8a691d56694e69f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:40.709013 19585 tablet_service.cc:1467] Processing CreateTablet for tablet ea0c6538a3c84b02b8a691d56694e69f (DEFAULT_TABLE table=test-table [id=cd71204ea6a54516b7c6eca1968b94e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:40.710563 19585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0c6538a3c84b02b8a691d56694e69f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:40.718235 19889 tablet_service.cc:1467] Processing CreateTablet for tablet ea0c6538a3c84b02b8a691d56694e69f (DEFAULT_TABLE table=test-table [id=cd71204ea6a54516b7c6eca1968b94e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:40.719939 19889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0c6538a3c84b02b8a691d56694e69f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:40.725965 19989 tablet_bootstrap.cc:492] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Bootstrap starting.
I20260812 02:42:40.727982 19990 tablet_bootstrap.cc:492] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: Bootstrap starting.
I20260812 02:42:40.730173 19989 tablet_bootstrap.cc:654] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:40.731184 19990 tablet_bootstrap.cc:654] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:40.731571 19989 log.cc:824] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:40.732360 19990 log.cc:824] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:40.734902 19989 tablet_bootstrap.cc:492] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: No bootstrap required, opened a new log
I20260812 02:42:40.735199 19989 ts_tablet_manager.cc:1397] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.001s
I20260812 02:42:40.741205 19990 tablet_bootstrap.cc:492] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: No bootstrap required, opened a new log
I20260812 02:42:40.741422 19993 tablet_bootstrap.cc:492] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: Bootstrap starting.
I20260812 02:42:40.741557 19990 ts_tablet_manager.cc:1397] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260812 02:42:40.744254 19989 raft_consensus.cc:348] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.744699 19989 raft_consensus.cc:374] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:40.744854 19989 raft_consensus.cc:729] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e7ed64d69f44efa05d46b34a718412, State: Initialized, Role: FOLLOWER
I20260812 02:42:40.745255 19993 tablet_bootstrap.cc:654] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:40.745615 19989 consensus_queue.cc:260] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.746623 19993 log.cc:824] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:40.746886 19816 heartbeater.cc:503] Master 127.17.196.254:42451 was elected leader, sending a full tablet report...
I20260812 02:42:40.747537 19989 ts_tablet_manager.cc:1428] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
I20260812 02:42:40.747740 19994 raft_consensus.cc:482] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:40.748008 19994 raft_consensus.cc:504] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.750033 19994 leader_election.cc:288] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479), 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365)
I20260812 02:42:40.753502 19990 raft_consensus.cc:348] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.754184 19990 raft_consensus.cc:374] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:40.754343 19990 raft_consensus.cc:729] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74730d68fe99401aa2633bd96391bf31, State: Initialized, Role: FOLLOWER
I20260812 02:42:40.757105 19993 tablet_bootstrap.cc:492] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: No bootstrap required, opened a new log
I20260812 02:42:40.757431 19993 ts_tablet_manager.cc:1397] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.000s
I20260812 02:42:40.757088 19990 consensus_queue.cc:260] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.763863 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.751147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:42289 (local address 127.17.196.195:36365)
0812 02:42:40.751381 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.751389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.756293 (+ 4904us) server_negotiation.cc:407] Connection header received
0812 02:42:40.756504 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.756512 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.756590 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.756684 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:40.757441 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.758541 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.761177 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.762505 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.762668 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.762774 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.763390 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.763551 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:42:40.764952 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.750731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:36365 (local address 127.17.196.194:42289)
0812 02:42:40.756160 (+ 5429us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:40.756177 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:40.756333 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:40.756851 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:40.756855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:40.756880 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:40.757265 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.757277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.758733 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.758740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.761003 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.761016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.763005 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.763011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.763188 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.763224 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 02:42:40.764349 (+ 1125us) client_negotiation.cc:240] Negotiation successful
0812 02:42:40.764522 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5291,"thread_start_us":4097,"threads_started":1}
I20260812 02:42:40.765285 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.750914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:54725 (local address 127.17.196.193:35479)
0812 02:42:40.751134 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.751142 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.755372 (+ 4230us) server_negotiation.cc:407] Connection header received
0812 02:42:40.755737 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.755746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.755855 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.755959 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:40.756876 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.758059 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.761003 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.764014 (+ 3011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.764182 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.764259 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.764865 (+ 606us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.765036 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:42:40.766208 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.750379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:35479 (local address 127.17.196.194:54725)
0812 02:42:40.755228 (+ 4849us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:40.755249 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:40.755424 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:40.756143 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:40.756146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:40.756170 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:40.756701 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.756718 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.758215 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.758221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.760792 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:40.760810 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.764471 (+ 3661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.764478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:40.764665 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.764699 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:42:40.765617 (+ 918us) client_negotiation.cc:240] Negotiation successful
0812 02:42:40.765776 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4646,"mutex_wait_us":32,"thread_start_us":5466,"threads_started":1}
I20260812 02:42:40.766559 19917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0c6538a3c84b02b8a691d56694e69f" candidate_uuid: "45e7ed64d69f44efa05d46b34a718412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" is_pre_election: true
W20260812 02:42:40.767441 19817 tablet.cc:2369] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:42:40.767311 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0c6538a3c84b02b8a691d56694e69f" candidate_uuid: "45e7ed64d69f44efa05d46b34a718412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74730d68fe99401aa2633bd96391bf31" is_pre_election: true
I20260812 02:42:40.767988 19614 raft_consensus.cc:2454] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45e7ed64d69f44efa05d46b34a718412 in term 0.
I20260812 02:42:40.768666 19690 leader_election.cc:302] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e7ed64d69f44efa05d46b34a718412, 74730d68fe99401aa2633bd96391bf31; no voters:
W20260812 02:42:40.768998 19689 leader_election.cc:341] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365): Illegal state: must be running to vote when last-logged opid is not known
I20260812 02:42:40.769122 19994 raft_consensus.cc:2781] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:40.769323 19994 raft_consensus.cc:482] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:40.769511 19994 raft_consensus.cc:3037] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:40.771760 19994 raft_consensus.cc:504] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.772503 19994 leader_election.cc:288] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 election: Requested vote from peers 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479), 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365)
I20260812 02:42:40.773047 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0c6538a3c84b02b8a691d56694e69f" candidate_uuid: "45e7ed64d69f44efa05d46b34a718412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74730d68fe99401aa2633bd96391bf31"
I20260812 02:42:40.773196 19917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0c6538a3c84b02b8a691d56694e69f" candidate_uuid: "45e7ed64d69f44efa05d46b34a718412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4c2bc2fb5745cfb066cb5d25554260"
I20260812 02:42:40.773319 19614 raft_consensus.cc:3037] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Advancing to term 1
W20260812 02:42:40.773856 19689 leader_election.cc:341] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365): Illegal state: must be running to vote when last-logged opid is not known
I20260812 02:42:40.774744 19990 ts_tablet_manager.cc:1428] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: Time spent starting tablet: real 0.033s user 0.015s sys 0.013s
I20260812 02:42:40.776366 19614 raft_consensus.cc:2454] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e7ed64d69f44efa05d46b34a718412 in term 1.
I20260812 02:42:40.776546 19993 raft_consensus.cc:348] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.776921 19690 leader_election.cc:302] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45e7ed64d69f44efa05d46b34a718412, 74730d68fe99401aa2633bd96391bf31; no voters: 0c4c2bc2fb5745cfb066cb5d25554260
I20260812 02:42:40.777066 19993 raft_consensus.cc:374] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:40.777192 19993 raft_consensus.cc:729] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4c2bc2fb5745cfb066cb5d25554260, State: Initialized, Role: FOLLOWER
I20260812 02:42:40.777571 19994 raft_consensus.cc:2781] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:40.777920 19993 consensus_queue.cc:260] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.779054 19969 heartbeater.cc:503] Master 127.17.196.254:42451 was elected leader, sending a full tablet report...
I20260812 02:42:40.779721 19994 raft_consensus.cc:686] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 1 LEADER]: Becoming Leader. State: Replica: 45e7ed64d69f44efa05d46b34a718412, State: Running, Role: LEADER
I20260812 02:42:40.780337 19994 consensus_queue.cc:237] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.781330 19993 ts_tablet_manager.cc:1428] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: Time spent starting tablet: real 0.024s user 0.012s sys 0.004s
I20260812 02:42:40.785687 19435 catalog_manager.cc:5495] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45e7ed64d69f44efa05d46b34a718412 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "45e7ed64d69f44efa05d46b34a718412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } health_report { overall_health: UNKNOWN } } }
I20260812 02:42:40.821066 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:40.814739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.17.196.194:35317)
0812 02:42:40.814955 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:40.814960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:40.815211 (+ 251us) server_negotiation.cc:407] Connection header received
0812 02:42:40.815554 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:40.815559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:40.815645 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:40.815789 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:40.816228 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.817157 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.818702 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:40.819150 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:40.819252 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:40.819834 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:40.820438 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:40.820576 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:40.820729 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 02:42:40.820805 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:42:40.830042 19917 raft_consensus.cc:3037] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:40.830385 19614 raft_consensus.cc:1261] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Refusing update from remote peer 45e7ed64d69f44efa05d46b34a718412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:40.831447 20000 consensus_queue.cc:1035] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:40.833190 19917 raft_consensus.cc:1261] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Refusing update from remote peer 45e7ed64d69f44efa05d46b34a718412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:40.834110 20000 consensus_queue.cc:1035] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:40.868752 19436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:40.869071 19436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:40.873684 19436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e: Sending DeleteTablet for 3 replicas of tablet ea0c6538a3c84b02b8a691d56694e69f
I20260812 02:42:40.875213 19585 tablet_service.cc:1514] Processing DeleteTablet for tablet ea0c6538a3c84b02b8a691d56694e69f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:40 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:34094
I20260812 02:42:40.875312 19889 tablet_service.cc:1514] Processing DeleteTablet for tablet ea0c6538a3c84b02b8a691d56694e69f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:40 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:53804
I20260812 02:42:40.876672 19741 tablet_service.cc:1514] Processing DeleteTablet for tablet ea0c6538a3c84b02b8a691d56694e69f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:40 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:42648
I20260812 02:42:40.876904 19435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:42:40.877689 19435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:40.879568 20018 tablet_replica.cc:330] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: stopping tablet replica
I20260812 02:42:40.880087 20018 raft_consensus.cc:2229] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:42:40.880970 20018 raft_consensus.cc:2258] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:40.881892 20016 tablet_replica.cc:330] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: stopping tablet replica
I20260812 02:42:40.882377 20016 raft_consensus.cc:2229] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:40.882992 20016 raft_consensus.cc:2258] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:40.885424 20016 ts_tablet_manager.cc:1905] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:40.894137 20017 tablet_replica.cc:330] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: stopping tablet replica
I20260812 02:42:40.894677 20017 raft_consensus.cc:2229] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:40.895278 20017 raft_consensus.cc:2258] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:40.895401 19741 tablet_service.cc:1467] Processing CreateTablet for tablet 8178d1c947d04122bbf1e715a62ab9ea (DEFAULT_TABLE table=test-table [id=0d3920a5b9974ab1bfb58d5ac3aa6d92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:40.895502 19889 tablet_service.cc:1467] Processing CreateTablet for tablet 8178d1c947d04122bbf1e715a62ab9ea (DEFAULT_TABLE table=test-table [id=0d3920a5b9974ab1bfb58d5ac3aa6d92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:40.895675 20016 ts_tablet_manager.cc:1918] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:42:40.895879 20016 log.cc:1196] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/ea0c6538a3c84b02b8a691d56694e69f
I20260812 02:42:40.895957 19889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8178d1c947d04122bbf1e715a62ab9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:40.896004 19585 tablet_service.cc:1467] Processing CreateTablet for tablet 8178d1c947d04122bbf1e715a62ab9ea (DEFAULT_TABLE table=test-table [id=0d3920a5b9974ab1bfb58d5ac3aa6d92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:40.896528 19585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8178d1c947d04122bbf1e715a62ab9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:40.896670 20016 ts_tablet_manager.cc:1939] T ea0c6538a3c84b02b8a691d56694e69f P 0c4c2bc2fb5745cfb066cb5d25554260: Deleting consensus metadata
I20260812 02:42:40.898663 20017 ts_tablet_manager.cc:1905] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:40.899049 19422 catalog_manager.cc:4841] TS 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365): tablet ea0c6538a3c84b02b8a691d56694e69f (table test-table [id=cd71204ea6a54516b7c6eca1968b94e1]) successfully deleted
I20260812 02:42:40.895581 20018 ts_tablet_manager.cc:1905] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:40.895931 19741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8178d1c947d04122bbf1e715a62ab9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:40.905890 19993 tablet_bootstrap.cc:492] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: Bootstrap starting.
I20260812 02:42:40.907907 19993 tablet_bootstrap.cc:654] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:40.908016 19989 tablet_bootstrap.cc:492] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: Bootstrap starting.
I20260812 02:42:40.908555 20018 ts_tablet_manager.cc:1918] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:42:40.908762 20018 log.cc:1196] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/ea0c6538a3c84b02b8a691d56694e69f
I20260812 02:42:40.910091 19989 tablet_bootstrap.cc:654] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:40.910794 20018 ts_tablet_manager.cc:1939] T ea0c6538a3c84b02b8a691d56694e69f P 45e7ed64d69f44efa05d46b34a718412: Deleting consensus metadata
I20260812 02:42:40.913012 19422 catalog_manager.cc:4841] TS 45e7ed64d69f44efa05d46b34a718412 (127.17.196.194:35317): tablet ea0c6538a3c84b02b8a691d56694e69f (table test-table [id=cd71204ea6a54516b7c6eca1968b94e1]) successfully deleted
I20260812 02:42:40.914201 19990 tablet_bootstrap.cc:492] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: Bootstrap starting.
I20260812 02:42:40.914767 20017 ts_tablet_manager.cc:1918] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:42:40.914988 20017 log.cc:1196] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/ea0c6538a3c84b02b8a691d56694e69f
I20260812 02:42:40.915712 20017 ts_tablet_manager.cc:1939] T ea0c6538a3c84b02b8a691d56694e69f P 74730d68fe99401aa2633bd96391bf31: Deleting consensus metadata
I20260812 02:42:40.916733 19990 tablet_bootstrap.cc:654] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:40.917902 19423 catalog_manager.cc:4841] TS 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479): tablet ea0c6538a3c84b02b8a691d56694e69f (table test-table [id=cd71204ea6a54516b7c6eca1968b94e1]) successfully deleted
I20260812 02:42:40.923659 19989 tablet_bootstrap.cc:492] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: No bootstrap required, opened a new log
I20260812 02:42:40.924057 19989 ts_tablet_manager.cc:1397] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.001s
I20260812 02:42:40.927297 19989 raft_consensus.cc:348] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.927845 19989 raft_consensus.cc:374] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:40.928049 19989 raft_consensus.cc:729] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e7ed64d69f44efa05d46b34a718412, State: Initialized, Role: FOLLOWER
I20260812 02:42:40.928532 19989 consensus_queue.cc:260] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.928882 19990 tablet_bootstrap.cc:492] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: No bootstrap required, opened a new log
I20260812 02:42:40.929458 19989 ts_tablet_manager.cc:1428] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 02:42:40.930114 19990 ts_tablet_manager.cc:1397] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.013s
I20260812 02:42:40.931128 19990 raft_consensus.cc:348] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.931407 19990 raft_consensus.cc:374] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:40.931519 19990 raft_consensus.cc:729] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74730d68fe99401aa2633bd96391bf31, State: Initialized, Role: FOLLOWER
I20260812 02:42:40.931926 19993 tablet_bootstrap.cc:492] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: No bootstrap required, opened a new log
I20260812 02:42:40.931818 19990 consensus_queue.cc:260] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.932978 19993 ts_tablet_manager.cc:1397] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.008s
I20260812 02:42:40.934059 19993 raft_consensus.cc:348] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.934481 19993 raft_consensus.cc:374] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:40.934676 19993 raft_consensus.cc:729] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4c2bc2fb5745cfb066cb5d25554260, State: Initialized, Role: FOLLOWER
I20260812 02:42:40.935079 19993 consensus_queue.cc:260] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:40.936100 19990 ts_tablet_manager.cc:1428] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260812 02:42:40.936404 19993 ts_tablet_manager.cc:1428] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260812 02:42:40.950266 19666 tablet.cc:2369] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:42:41.071369 20009 raft_consensus.cc:482] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:41.071715 20009 raft_consensus.cc:504] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.072686 20009 leader_election.cc:288] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479), 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365)
I20260812 02:42:41.073385 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8178d1c947d04122bbf1e715a62ab9ea" candidate_uuid: "45e7ed64d69f44efa05d46b34a718412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74730d68fe99401aa2633bd96391bf31" is_pre_election: true
I20260812 02:42:41.073462 19917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8178d1c947d04122bbf1e715a62ab9ea" candidate_uuid: "45e7ed64d69f44efa05d46b34a718412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" is_pre_election: true
I20260812 02:42:41.073730 19614 raft_consensus.cc:2454] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45e7ed64d69f44efa05d46b34a718412 in term 0.
I20260812 02:42:41.073879 19917 raft_consensus.cc:2454] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45e7ed64d69f44efa05d46b34a718412 in term 0.
I20260812 02:42:41.074316 19690 leader_election.cc:302] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e7ed64d69f44efa05d46b34a718412, 74730d68fe99401aa2633bd96391bf31; no voters:
I20260812 02:42:41.074649 20009 raft_consensus.cc:2781] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:41.074810 20009 raft_consensus.cc:482] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:41.074945 20009 raft_consensus.cc:3037] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:41.077658 20009 raft_consensus.cc:504] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.078402 20009 leader_election.cc:288] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 election: Requested vote from peers 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479), 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365)
I20260812 02:42:41.079048 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8178d1c947d04122bbf1e715a62ab9ea" candidate_uuid: "45e7ed64d69f44efa05d46b34a718412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74730d68fe99401aa2633bd96391bf31"
I20260812 02:42:41.079099 19917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8178d1c947d04122bbf1e715a62ab9ea" candidate_uuid: "45e7ed64d69f44efa05d46b34a718412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4c2bc2fb5745cfb066cb5d25554260"
I20260812 02:42:41.079290 19614 raft_consensus.cc:3037] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:41.079334 19917 raft_consensus.cc:3037] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:41.081735 19614 raft_consensus.cc:2454] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e7ed64d69f44efa05d46b34a718412 in term 1.
I20260812 02:42:41.082288 19690 leader_election.cc:302] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e7ed64d69f44efa05d46b34a718412, 74730d68fe99401aa2633bd96391bf31; no voters:
I20260812 02:42:41.082500 19917 raft_consensus.cc:2454] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e7ed64d69f44efa05d46b34a718412 in term 1.
I20260812 02:42:41.082736 20009 raft_consensus.cc:2781] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:41.082970 20009 raft_consensus.cc:686] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 1 LEADER]: Becoming Leader. State: Replica: 45e7ed64d69f44efa05d46b34a718412, State: Running, Role: LEADER
I20260812 02:42:41.083356 20009 consensus_queue.cc:237] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.086607 19434 catalog_manager.cc:5495] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45e7ed64d69f44efa05d46b34a718412 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "45e7ed64d69f44efa05d46b34a718412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } health_report { overall_health: UNKNOWN } } }
W20260812 02:42:41.097326 19970 tablet.cc:2369] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 02:42:41 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502558, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:41.145363 19846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.128282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.17.196.254:42451)
0812 02:42:41.128463 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.128468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.129307 (+ 839us) server_negotiation.cc:407] Connection header received
0812 02:42:41.129518 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.129525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.129608 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.129721 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:41.131072 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.132243 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.133771 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.134519 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.134658 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.138472 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:41.138479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:41.138484 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:41.138546 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:41.142232 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.143072 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.143078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.143083 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.143245 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.143618 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.143623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.143627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.144055 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:41.144201 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.144706 (+ 505us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.144977 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 02:42:41.145440 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.128377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42451 (local address 127.0.0.1:58856)
0812 02:42:41.129171 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:41.129194 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:41.129358 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:41.129763 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:41.129767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:41.130625 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:41.130910 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.130920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.132407 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.132413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.133623 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.133635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.134749 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.134754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.134875 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.135616 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:41.135634 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:41.138269 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:41.142413 (+ 4144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:41.142428 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:41.142435 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:41.142946 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:41.143360 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:41.143364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:41.143366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:41.143507 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:41.144195 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:41.144203 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:41.144423 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 02:42:41.144565 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 02:42:41.144805 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":392,"threads_started":1}
I20260812 02:42:41.151512 19434 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:58856:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 02:42:41.152711 19434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:41.160094 19585 tablet_service.cc:1467] Processing CreateTablet for tablet 9484ef8c1ae34c8d9056cdaa977d9b3d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35e8c3dd153549258f0e1fa2876e22ad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:42:41.160238 19741 tablet_service.cc:1467] Processing CreateTablet for tablet 9484ef8c1ae34c8d9056cdaa977d9b3d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35e8c3dd153549258f0e1fa2876e22ad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:42:41.160754 19585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9484ef8c1ae34c8d9056cdaa977d9b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:41.160518 19889 tablet_service.cc:1467] Processing CreateTablet for tablet 9484ef8c1ae34c8d9056cdaa977d9b3d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35e8c3dd153549258f0e1fa2876e22ad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:42:41.160861 19741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9484ef8c1ae34c8d9056cdaa977d9b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:41.161128 19889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9484ef8c1ae34c8d9056cdaa977d9b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:41.168793 19990 tablet_bootstrap.cc:492] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: Bootstrap starting.
I20260812 02:42:41.168916 19989 tablet_bootstrap.cc:492] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: Bootstrap starting.
I20260812 02:42:41.170305 19990 tablet_bootstrap.cc:654] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:41.170751 19989 tablet_bootstrap.cc:654] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:41.171715 19993 tablet_bootstrap.cc:492] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: Bootstrap starting.
I20260812 02:42:41.174196 19993 tablet_bootstrap.cc:654] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:41.181396 19989 tablet_bootstrap.cc:492] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: No bootstrap required, opened a new log
I20260812 02:42:41.181664 19989 ts_tablet_manager.cc:1397] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260812 02:42:41.182626 19989 raft_consensus.cc:348] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.182904 19989 raft_consensus.cc:374] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:41.183007 19989 raft_consensus.cc:729] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e7ed64d69f44efa05d46b34a718412, State: Initialized, Role: FOLLOWER
I20260812 02:42:41.183331 19989 consensus_queue.cc:260] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.183975 19989 ts_tablet_manager.cc:1428] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 02:42:41.183836 20009 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.184067 19990 tablet_bootstrap.cc:492] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: No bootstrap required, opened a new log
I20260812 02:42:41.184177 20009 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:41.184249 19990 ts_tablet_manager.cc:1397] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260812 02:42:41.185020 19990 raft_consensus.cc:348] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.185277 19990 raft_consensus.cc:374] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:41.185365 19990 raft_consensus.cc:729] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74730d68fe99401aa2633bd96391bf31, State: Initialized, Role: FOLLOWER
I20260812 02:42:41.185655 19990 consensus_queue.cc:260] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.186249 19998 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.186770 19998 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:41.187026 19990 ts_tablet_manager.cc:1428] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 02:42:41.189302 19993 tablet_bootstrap.cc:492] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: No bootstrap required, opened a new log
I20260812 02:42:41.189589 19993 ts_tablet_manager.cc:1397] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.004s
I20260812 02:42:41.191349 19993 raft_consensus.cc:348] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.191612 19993 raft_consensus.cc:374] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:41.191694 19993 raft_consensus.cc:729] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4c2bc2fb5745cfb066cb5d25554260, State: Initialized, Role: FOLLOWER
I20260812 02:42:41.192001 19993 consensus_queue.cc:260] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.192660 19993 ts_tablet_manager.cc:1428] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:42:41.192554 19999 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.192881 19999 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:41.270642 19999 raft_consensus.cc:482] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:41.270901 19999 raft_consensus.cc:504] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.272907 19999 leader_election.cc:288] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479), 45e7ed64d69f44efa05d46b34a718412 (127.17.196.194:35317)
I20260812 02:42:41.282056 20034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.273214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:35479 (local address 127.17.196.195:43137)
0812 02:42:41.273972 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:41.273995 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:41.274148 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:41.274957 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:41.274964 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:41.275018 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:41.275340 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.275351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.276777 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.276783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.279001 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.279018 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.281010 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.281017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.281253 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.281292 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:42:41.281403 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 02:42:41.281603 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":1,"thread_start_us":617,"threads_started":1}
I20260812 02:42:41.282328 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.273307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:33427 (local address 127.17.196.194:35317)
0812 02:42:41.273547 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.273555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.275149 (+ 1594us) server_negotiation.cc:407] Connection header received
0812 02:42:41.275339 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.275349 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.275428 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.275532 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:41.276121 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.277326 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.279718 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.280873 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.281037 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.281115 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.281681 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.281846 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:42:41.282928 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.273225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35317 (local address 127.17.196.195:33427)
0812 02:42:41.274997 (+ 1772us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:41.275026 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:41.275171 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:41.275559 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:41.275567 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:41.275603 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:41.275903 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.275918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.277521 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.277527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.279550 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.279562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.281243 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.281250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.281451 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.281508 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 02:42:41.282397 (+ 889us) client_negotiation.cc:240] Negotiation successful
0812 02:42:41.282547 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1628,"mutex_wait_us":259,"thread_start_us":1124,"threads_started":1}
I20260812 02:42:41.282980 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.273701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:43137 (local address 127.17.196.193:35479)
0812 02:42:41.274608 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.274615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.274635 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:42:41.274717 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.274721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.274801 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.274916 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:41.275507 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.276605 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.279201 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.280496 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.280700 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.280781 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.282515 (+ 1734us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.282690 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":673,"threads_started":1}
I20260812 02:42:41.283623 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9484ef8c1ae34c8d9056cdaa977d9b3d" candidate_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74730d68fe99401aa2633bd96391bf31" is_pre_election: true
I20260812 02:42:41.283952 19614 raft_consensus.cc:2454] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4c2bc2fb5745cfb066cb5d25554260 in term 0.
I20260812 02:42:41.284096 19762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9484ef8c1ae34c8d9056cdaa977d9b3d" candidate_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e7ed64d69f44efa05d46b34a718412" is_pre_election: true
I20260812 02:42:41.284565 19762 raft_consensus.cc:2454] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4c2bc2fb5745cfb066cb5d25554260 in term 0.
I20260812 02:42:41.284554 19840 leader_election.cc:302] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c2bc2fb5745cfb066cb5d25554260, 74730d68fe99401aa2633bd96391bf31; no voters:
I20260812 02:42:41.285063 19999 raft_consensus.cc:2781] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:41.285209 19999 raft_consensus.cc:482] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:41.285320 19999 raft_consensus.cc:3037] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:41.288723 19999 raft_consensus.cc:504] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.289520 19999 leader_election.cc:288] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [CANDIDATE]: Term 1 election: Requested vote from peers 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479), 45e7ed64d69f44efa05d46b34a718412 (127.17.196.194:35317)
I20260812 02:42:41.290103 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9484ef8c1ae34c8d9056cdaa977d9b3d" candidate_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74730d68fe99401aa2633bd96391bf31"
I20260812 02:42:41.290371 19614 raft_consensus.cc:3037] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:41.290661 19762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9484ef8c1ae34c8d9056cdaa977d9b3d" candidate_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e7ed64d69f44efa05d46b34a718412"
I20260812 02:42:41.290897 19762 raft_consensus.cc:3037] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:41.293215 19762 raft_consensus.cc:2454] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4c2bc2fb5745cfb066cb5d25554260 in term 1.
I20260812 02:42:41.293442 19614 raft_consensus.cc:2454] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4c2bc2fb5745cfb066cb5d25554260 in term 1.
I20260812 02:42:41.293841 19839 leader_election.cc:302] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c2bc2fb5745cfb066cb5d25554260, 45e7ed64d69f44efa05d46b34a718412; no voters:
I20260812 02:42:41.294226 19999 raft_consensus.cc:2781] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:41.294730 19999 raft_consensus.cc:686] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 LEADER]: Becoming Leader. State: Replica: 0c4c2bc2fb5745cfb066cb5d25554260, State: Running, Role: LEADER
I20260812 02:42:41.295156 19999 consensus_queue.cc:237] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } }
I20260812 02:42:41.297797 20037 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: TxnStatusTablet state changed. Reason: New leader 0c4c2bc2fb5745cfb066cb5d25554260. Latest consensus state: current_term: 1 leader_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.298171 20037 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:42:41.299360 20039 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 02:42:41.299288 19434 catalog_manager.cc:5495] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } health_report { overall_health: HEALTHY } } }
I20260812 02:42:41.299754 20039 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 02:42:41.324980 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.315566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:36365 (local address 127.0.0.1:53816)
0812 02:42:41.315979 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:41.316002 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:41.316154 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:41.317422 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:41.317429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:41.317444 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:42:41.317713 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.317728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.319688 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.319697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.321340 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.321359 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.322550 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.322558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.322796 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.322907 (+ 111us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:42:41.324297 (+ 1390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:42:41.324341 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 02:42:41.324410 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 02:42:41.324589 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260812 02:42:41.326198 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.316089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.17.196.195:36365)
0812 02:42:41.316925 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.316933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.316961 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:42:41.317064 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.317069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.317191 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.317880 (+ 689us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:41.318207 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.319493 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.321585 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.322339 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.323123 (+ 784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.323292 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:41.324140 (+ 848us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:41.325606 (+ 1466us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.325678 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.325773 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":513,"threads_started":1}
I20260812 02:42:41.331945 19614 raft_consensus.cc:1261] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Refusing update from remote peer 0c4c2bc2fb5745cfb066cb5d25554260: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:41.332248 19762 raft_consensus.cc:1261] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412 [term 1 FOLLOWER]: Refusing update from remote peer 0c4c2bc2fb5745cfb066cb5d25554260: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:41.332937 20037 consensus_queue.cc:1035] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:41.333361 19999 consensus_queue.cc:1035] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:41.346298 19998 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: TxnStatusTablet state changed. Reason: New leader 0c4c2bc2fb5745cfb066cb5d25554260. Latest consensus state: current_term: 1 leader_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.346289 20009 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: TxnStatusTablet state changed. Reason: New leader 0c4c2bc2fb5745cfb066cb5d25554260. Latest consensus state: current_term: 1 leader_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.346654 19998 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:41.346686 20009 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:41.351681 20005 mvcc.cc:205] Tried to move back new op lower bound from 7317514491206074368 to 7317514491068940288. Current Snapshot: MvccSnapshot[applied={T|T < 7317514491206074368}]
I20260812 02:42:41.353864 20004 mvcc.cc:205] Tried to move back new op lower bound from 7317514491206074368 to 7317514491068940288. Current Snapshot: MvccSnapshot[applied={T|T < 7317514491206074368}]
I20260812 02:42:41.355772 20003 mvcc.cc:205] Tried to move back new op lower bound from 7317514491206074368 to 7317514491068940288. Current Snapshot: MvccSnapshot[applied={T|T < 7317514491206074368}]
I20260812 02:42:41.362087 19999 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.362531 19999 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:42:41.366451 19999 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.366822 19999 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 0c4c2bc2fb5745cfb066cb5d25554260: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:42:41.372382 19998 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.372843 19998 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 74730d68fe99401aa2633bd96391bf31: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:41.380324 20009 tablet_replica.cc:439] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 } } peers { permanent_uuid: "45e7ed64d69f44efa05d46b34a718412" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35317 } } peers { permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 } } }
I20260812 02:42:41.380688 20009 tablet_replica.cc:442] T 9484ef8c1ae34c8d9056cdaa977d9b3d P 45e7ed64d69f44efa05d46b34a718412: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:41.397262 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.391202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:56935 (local address 127.17.196.195:36365)
0812 02:42:41.391396 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.391403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.392061 (+ 658us) server_negotiation.cc:407] Connection header received
0812 02:42:41.392303 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.392309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.392389 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.392495 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:41.393065 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.393923 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.395405 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.395866 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.396015 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.396482 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:41.396600 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:41.396702 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.396856 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.396941 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 02:42:41.397549 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.391188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:36365 (local address 127.17.196.194:56935)
0812 02:42:41.391937 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:41.391955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:41.392144 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:41.392514 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:41.392519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:41.392552 (+ 33us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:42:41.392897 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.392907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.394082 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.394088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.395255 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.395268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.396074 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.396082 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.396228 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.396288 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:42:41.396707 (+ 419us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:42:41.396725 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 02:42:41.396832 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 02:42:41.396982 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":409,"threads_started":1}
I20260812 02:42:41.409685 19917 raft_consensus.cc:1261] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Refusing update from remote peer 45e7ed64d69f44efa05d46b34a718412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:41.409816 19614 raft_consensus.cc:1261] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Refusing update from remote peer 45e7ed64d69f44efa05d46b34a718412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:41.410343 20009 consensus_queue.cc:1035] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c4c2bc2fb5745cfb066cb5d25554260" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:41.410859 20009 consensus_queue.cc:1035] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74730d68fe99401aa2633bd96391bf31" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:41.461241 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.451527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:44997 (local address 127.17.196.194:35317)
0812 02:42:41.451801 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.451811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.453037 (+ 1226us) server_negotiation.cc:407] Connection header received
0812 02:42:41.453394 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.453400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.453552 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.453628 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:41.454664 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.455922 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.458458 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.459172 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.459302 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.460178 (+ 876us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:41.460329 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:41.460469 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.460744 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.460838 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260812 02:42:41.461634 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.452062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35317 (local address 127.17.196.195:44997)
0812 02:42:41.452921 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:41.452944 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:41.453180 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:41.453914 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:41.453920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:41.453935 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:42:41.454392 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.454407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.456195 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.456199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.458282 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:41.458296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.459709 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.459714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:41.459848 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.459908 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:42:41.460485 (+ 577us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:42:41.460500 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 02:42:41.460714 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 02:42:41.460867 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":3731,"threads_started":1}
I20260812 02:42:41.505209 19434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:41.505483 19434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:41.508939 19434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d31bd36dee34d0a9d7c20ad11361a1e: Sending DeleteTablet for 3 replicas of tablet 8178d1c947d04122bbf1e715a62ab9ea
I20260812 02:42:41.509979 19585 tablet_service.cc:1514] Processing DeleteTablet for tablet 8178d1c947d04122bbf1e715a62ab9ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:41 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:34094
I20260812 02:42:41.510463 19889 tablet_service.cc:1514] Processing DeleteTablet for tablet 8178d1c947d04122bbf1e715a62ab9ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:41 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:53804
I20260812 02:42:41.510844 19741 tablet_service.cc:1514] Processing DeleteTablet for tablet 8178d1c947d04122bbf1e715a62ab9ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:41 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:42648
I20260812 02:42:41.517375 20059 tablet_replica.cc:330] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: stopping tablet replica
I20260812 02:42:41.517740 20059 raft_consensus.cc:2229] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:42:41.518329 20059 raft_consensus.cc:2258] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:41.521659 20059 ts_tablet_manager.cc:1905] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:41.524729 20057 tablet_replica.cc:330] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: stopping tablet replica
I20260812 02:42:41.525033 20057 raft_consensus.cc:2229] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:41.525363 20057 raft_consensus.cc:2258] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:41.527066 20057 ts_tablet_manager.cc:1905] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:41.532793 20059 ts_tablet_manager.cc:1918] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:41.532892 20058 tablet_replica.cc:330] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: stopping tablet replica
I20260812 02:42:41.533156 20059 log.cc:1196] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/8178d1c947d04122bbf1e715a62ab9ea
I20260812 02:42:41.533298 20058 raft_consensus.cc:2229] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:41.533749 20058 raft_consensus.cc:2258] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:41.533926 20059 ts_tablet_manager.cc:1939] T 8178d1c947d04122bbf1e715a62ab9ea P 45e7ed64d69f44efa05d46b34a718412: Deleting consensus metadata
I20260812 02:42:41.535205 19422 catalog_manager.cc:4841] TS 45e7ed64d69f44efa05d46b34a718412 (127.17.196.194:35317): tablet 8178d1c947d04122bbf1e715a62ab9ea (table test-table [id=0d3920a5b9974ab1bfb58d5ac3aa6d92]) successfully deleted
I20260812 02:42:41.535328 20058 ts_tablet_manager.cc:1905] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:41.542047 20057 ts_tablet_manager.cc:1918] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:41.542335 20057 log.cc:1196] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/8178d1c947d04122bbf1e715a62ab9ea
I20260812 02:42:41.542857 20057 ts_tablet_manager.cc:1939] T 8178d1c947d04122bbf1e715a62ab9ea P 74730d68fe99401aa2633bd96391bf31: Deleting consensus metadata
I20260812 02:42:41.544281 19423 catalog_manager.cc:4841] TS 74730d68fe99401aa2633bd96391bf31 (127.17.196.193:35479): tablet 8178d1c947d04122bbf1e715a62ab9ea (table test-table [id=0d3920a5b9974ab1bfb58d5ac3aa6d92]) successfully deleted
Aug 12 02:42:41 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:42:41.545212 20058 ts_tablet_manager.cc:1918] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:41.545395 20058 log.cc:1196] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/8178d1c947d04122bbf1e715a62ab9ea
I20260812 02:42:41.546000 20058 ts_tablet_manager.cc:1939] T 8178d1c947d04122bbf1e715a62ab9ea P 0c4c2bc2fb5745cfb066cb5d25554260: Deleting consensus metadata
I20260812 02:42:41.547386 19422 catalog_manager.cc:4841] TS 0c4c2bc2fb5745cfb066cb5d25554260 (127.17.196.195:36365): tablet 8178d1c947d04122bbf1e715a62ab9ea (table test-table [id=0d3920a5b9974ab1bfb58d5ac3aa6d92]) successfully deleted
I20260812 02:42:41.553752 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.523270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.17.196.193:35479)
0812 02:42:41.523553 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.523563 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.531225 (+ 7662us) server_negotiation.cc:407] Connection header received
0812 02:42:41.531740 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.531750 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.531862 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.531949 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:41.534997 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.536168 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.538327 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.539118 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.539231 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.546017 (+ 6786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:41.546038 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:41.546060 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:41.546108 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:41.550310 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.551250 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.551261 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.551265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.551447 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.551973 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.551978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.551982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.552487 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:41.552817 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.553140 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.553363 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 02:42:41.554711 19660 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34098
I20260812 02:42:41.572023 19846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.558253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.17.196.254:42451)
0812 02:42:41.558518 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.558527 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.558739 (+ 212us) server_negotiation.cc:407] Connection header received
0812 02:42:41.558950 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.558955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.559025 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.559141 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:41.560744 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.561632 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.562901 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.563342 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.563499 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.566096 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:41.566101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:41.566104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:41.566141 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:41.569142 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.569957 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.569963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.569966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.570115 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.570508 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.570513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.570518 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.570819 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:41.571065 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.571437 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.571682 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
W20260812 02:42:41.572678 19434 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58862
I20260812 02:42:41.575835 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19511
I20260812 02:42:41.590691 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.576682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.17.196.193:35479)
0812 02:42:41.576877 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.576883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.577182 (+ 299us) server_negotiation.cc:407] Connection header received
0812 02:42:41.577325 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.577331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.577421 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.577573 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:41.578718 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.579870 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.581084 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.581762 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.581911 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.584247 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:41.584254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:41.584259 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:41.584301 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:41.587768 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.588445 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.588450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.588454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.588613 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.589075 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.589080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.589084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.589447 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:41.589720 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.590074 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.590326 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 02:42:41.591378 19660 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34110
W20260812 02:42:41.592051 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19511: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:41.592300 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 19511
I20260812 02:42:41.625151 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19670
Aug 12 02:42:41 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:42:41.644184 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.626091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.17.196.194:35317)
0812 02:42:41.626316 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.626322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.626634 (+ 312us) server_negotiation.cc:407] Connection header received
0812 02:42:41.626862 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.626868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.626958 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.627072 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:41.628422 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.629568 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.631040 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.631674 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.631808 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.636554 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:41.636574 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:41.636593 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:41.636640 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:41.640773 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.641783 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.641794 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.641798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.642000 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.642441 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.642448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.642453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.642946 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:41.643346 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.643504 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.643789 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 02:42:41.645057 19810 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42668
W20260812 02:42:41.645882 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19670: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:41.646085 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 19670
I20260812 02:42:41.678148 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19821
Aug 12 02:42:41 dist-test-slave-ss4n krb5kdc[19391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:42:41.698025 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:41.679078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53826 (local address 127.17.196.195:36365)
0812 02:42:41.679319 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:41.679326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:41.679347 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:42:41.679661 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:41.679673 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:41.679775 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:41.679928 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:41.681474 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.682443 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.683875 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:41.684403 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:41.684572 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:41.690013 (+ 5441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:41.690027 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:41.690043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:41.690091 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:41.694314 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.695351 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.695359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.695361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.695539 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:41.696016 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:41.696019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:41.696022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:41.696436 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:41.696767 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:41.697285 (+ 518us) server_negotiation.cc:299] Negotiation successful
0812 02:42:41.697557 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 02:42:41.698948 19964 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53826
W20260812 02:42:41.699721 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19821: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:41.699913 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 19821
I20260812 02:42:41.725713 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 19404
I20260812 02:42:41.727119 19479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:41.846606 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 19404
2026-08-12T02:42:41Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5087 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:41 dist-test-slave-ss4n krb5kdc[20069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:41 dist-test-slave-ss4n krb5kdc[20069](info): set up 2 sockets
Aug 12 02:42:41 dist-test-slave-ss4n krb5kdc[20069](info): commencing operation
krb5kdc: starting...
W20260812 02:42:43.093644 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:42:43 dist-test-slave-ss4n krb5kdc[20069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:42:43.334110 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:33733
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:33733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:43.683933 20084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:43.684303 20084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:43.684504 20084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:43.692695 20084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:43.692840 20084 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 02:42:43.692884 20084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:43.693034 20084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:43.693111 20084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:43.693233 20084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:43.703822 20084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:33733
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:33733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:43.705300 20084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:43.706841 20084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:43.722007 20091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:43.721947 20094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:43.722039 20092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:43.722990 20084 server_base.cc:1034] running on GCE node
I20260812 02:42:43.723950 20084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:43.725903 20084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:43.727272 20084 hybrid_clock.cc:648] HybridClock initialized: now 1786502563727144 us; error 46 us; skew 500 ppm
I20260812 02:42:43.727803 20084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:43 dist-test-slave-ss4n krb5kdc[20069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502563, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:43.731810 20084 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:42:43.733677 20084 webserver.cc:460] Webserver started at http://127.17.196.254:40765/ using document root <none> and password file <none>
I20260812 02:42:43.734510 20084 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:43.734663 20084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:43.735033 20084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:43.738745 20084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "1db2815f4f494995a73f5ab29fce1b06"
format_stamp: "Formatted at 2026-08-12 02:42:43 on dist-test-slave-ss4n"
server_key: "a20ff592b1b296d2f2da447656c855b4"
server_key_iv: "10bd981f8bc0791d33c9ff27227c2f61"
server_key_version: "encryptionkey@0"
I20260812 02:42:43.739696 20084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "1db2815f4f494995a73f5ab29fce1b06"
format_stamp: "Formatted at 2026-08-12 02:42:43 on dist-test-slave-ss4n"
server_key: "a20ff592b1b296d2f2da447656c855b4"
server_key_iv: "10bd981f8bc0791d33c9ff27227c2f61"
server_key_version: "encryptionkey@0"
I20260812 02:42:43.746222 20084 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:42:43.750473 20102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:43.752100 20084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260812 02:42:43.752332 20084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "1db2815f4f494995a73f5ab29fce1b06"
format_stamp: "Formatted at 2026-08-12 02:42:43 on dist-test-slave-ss4n"
server_key: "a20ff592b1b296d2f2da447656c855b4"
server_key_iv: "10bd981f8bc0791d33c9ff27227c2f61"
server_key_version: "encryptionkey@0"
I20260812 02:42:43.752674 20084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:43.768229 20084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:43.773552 20084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:43.773900 20084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:43.798384 20084 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:33733
I20260812 02:42:43.798393 20167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:33733 every 8 connection(s)
I20260812 02:42:43.799932 20084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:43.804183 20169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:43.809450 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 20084
I20260812 02:42:43.809760 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:42:43.810477 18195 external_mini_cluster.cc:1428] Setting key 8825dfb89b98bcf8d8f06e5c7ce27f9e
I20260812 02:42:43.816107 20169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: Bootstrap starting.
I20260812 02:42:43.820279 20169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:43.821796 20169 log.cc:824] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:43.825897 20169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: No bootstrap required, opened a new log
Aug 12 02:42:43 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:43.834561 20169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db2815f4f494995a73f5ab29fce1b06" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33733 } }
I20260812 02:42:43.834949 20169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:43.835143 20169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1db2815f4f494995a73f5ab29fce1b06, State: Initialized, Role: FOLLOWER
I20260812 02:42:43.835793 20169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db2815f4f494995a73f5ab29fce1b06" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33733 } }
I20260812 02:42:43.836090 20169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:43.836252 20169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:43.836469 20169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:43.838917 20173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:43.812850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.17.196.254:33733)
0812 02:42:43.813847 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:43.813863 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:43.813908 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:42:43.815747 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:43.815775 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:43.816224 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:43.816868 (+ 644us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:43.819004 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:43.820531 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:43.822593 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:43.823824 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:43.823988 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:43.829359 (+ 5371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:43.829368 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:43.829389 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:43.829443 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:43.833348 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:43.834319 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:43.834328 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:43.834336 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:43.834555 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:43.835056 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:43.835060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:43.835062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:43.835892 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:43.836271 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:43.836848 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 02:42:43.837385 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":452,"threads_started":1}
I20260812 02:42:43.839953 20169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db2815f4f494995a73f5ab29fce1b06" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33733 } }
I20260812 02:42:43.840518 20169 leader_election.cc:302] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1db2815f4f494995a73f5ab29fce1b06; no voters:
I20260812 02:42:43.841187 20169 leader_election.cc:288] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:43.841362 20175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:43.843595 20169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:43.845348 20175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [term 1 LEADER]: Becoming Leader. State: Replica: 1db2815f4f494995a73f5ab29fce1b06, State: Running, Role: LEADER
I20260812 02:42:43.846172 20175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db2815f4f494995a73f5ab29fce1b06" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33733 } }
I20260812 02:42:43.849421 20178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1db2815f4f494995a73f5ab29fce1b06. Latest consensus state: current_term: 1 leader_uuid: "1db2815f4f494995a73f5ab29fce1b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db2815f4f494995a73f5ab29fce1b06" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33733 } } }
I20260812 02:42:43.849579 20176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1db2815f4f494995a73f5ab29fce1b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db2815f4f494995a73f5ab29fce1b06" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33733 } } }
I20260812 02:42:43.849949 20178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:43.850030 20176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:43.851039 20190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:43.856982 20190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:43.867743 20190 catalog_manager.cc:1347] Generated new cluster ID: 73039e78c28147938fea4f5c0177c991
I20260812 02:42:43.867880 20190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:43.889142 20190 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:43.890646 20190 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:43.904379 20190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: Generated new TSK 0
I20260812 02:42:43.905395 20190 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:42:43.978814 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33733
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:44.353600 20200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:44.353919 20200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:44.354125 20200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:44.361879 20200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:44.362006 20200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:44.362174 20200 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 02:42:44.362241 20200 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:42:44.362349 20200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:42:44.372709 20200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:33733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:44.374274 20200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:44.375732 20200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:44.390604 20205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:44.390594 20208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:44.391171 20206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:44.391860 20200 server_base.cc:1034] running on GCE node
I20260812 02:42:44.392697 20200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:44.393980 20200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:44.395274 20200 hybrid_clock.cc:648] HybridClock initialized: now 1786502564395233 us; error 45 us; skew 500 ppm
I20260812 02:42:44.395735 20200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:44 dist-test-slave-ss4n krb5kdc[20069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502564, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:44.399672 20200 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:42:44.401414 20200 webserver.cc:460] Webserver started at http://127.17.196.193:41561/ using document root <none> and password file <none>
I20260812 02:42:44.402276 20200 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:44.402447 20200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:44.402797 20200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:44.406484 20200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "1dbc2147296446e58d6ad07c2851d0a4"
format_stamp: "Formatted at 2026-08-12 02:42:44 on dist-test-slave-ss4n"
server_key: "cdacb2f9238a9a102503719df769da83"
server_key_iv: "266c7adffaaa324e7dd4ded1a1a393b6"
server_key_version: "encryptionkey@0"
I20260812 02:42:44.407533 20200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "1dbc2147296446e58d6ad07c2851d0a4"
format_stamp: "Formatted at 2026-08-12 02:42:44 on dist-test-slave-ss4n"
server_key: "cdacb2f9238a9a102503719df769da83"
server_key_iv: "266c7adffaaa324e7dd4ded1a1a393b6"
server_key_version: "encryptionkey@0"
I20260812 02:42:44.413771 20200 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260812 02:42:44.418298 20215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:44.420574 20200 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:42:44.420821 20200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "1dbc2147296446e58d6ad07c2851d0a4"
format_stamp: "Formatted at 2026-08-12 02:42:44 on dist-test-slave-ss4n"
server_key: "cdacb2f9238a9a102503719df769da83"
server_key_iv: "266c7adffaaa324e7dd4ded1a1a393b6"
server_key_version: "encryptionkey@0"
I20260812 02:42:44.421077 20200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:44.447495 20200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:44.452996 20200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:44.453469 20200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:44.456838 20200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:44.457013 20200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:44.457186 20200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:44.457314 20200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 12 02:42:44 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502564, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:44.488368 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:44.457861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:48469 (local address 127.17.196.254:33733)
0812 02:42:44.458446 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:44.458455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:44.460120 (+ 1665us) server_negotiation.cc:407] Connection header received
0812 02:42:44.463266 (+ 3146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:44.463276 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:44.463396 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:44.463563 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:44.466638 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:44.468011 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:44.471591 (+ 3580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:44.472261 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:44.472439 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:44.479281 (+ 6842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:44.479294 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:44.479301 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:44.479370 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:44.483660 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:44.485198 (+ 1538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:44.485209 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:44.485215 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:44.485419 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:44.486183 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:44.486190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:44.486194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:44.486545 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:44.486753 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:44.487659 (+ 906us) server_negotiation.cc:299] Negotiation successful
0812 02:42:44.487952 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":320,"threads_started":1}
I20260812 02:42:44.489602 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:44.458767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33733 (local address 127.17.196.193:48469)
0812 02:42:44.459850 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:44.459934 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:44.462815 (+ 2881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:44.463927 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:44.463941 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:44.465024 (+ 1083us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:44.466217 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:44.466259 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:44.468300 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:44.468307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:44.471319 (+ 3012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:44.471349 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:44.472590 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:44.472600 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:44.473011 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:44.474139 (+ 1128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:44.474182 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:44.478836 (+ 4654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:44.483904 (+ 5068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:44.483914 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:44.483930 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:44.484976 (+ 1046us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:44.485634 (+ 658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:44.485648 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:44.485651 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:44.486006 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:44.486723 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:44.486733 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:44.487186 (+ 453us) client_negotiation.cc:769] Sending connection context
0812 02:42:44.487502 (+ 316us) client_negotiation.cc:240] Negotiation successful
0812 02:42:44.488147 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":465,"threads_started":1}
I20260812 02:42:44.517110 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:44.496876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:53539 (local address 127.17.196.254:33733)
0812 02:42:44.497110 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:44.497120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:44.498583 (+ 1463us) server_negotiation.cc:407] Connection header received
0812 02:42:44.498747 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:44.498752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:44.498825 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:44.498965 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:44.500701 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:44.501919 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:44.504643 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:44.505404 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:44.505682 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:44.509327 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:44.509334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:44.509338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:44.509380 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:44.513588 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:44.514524 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:44.514532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:44.514537 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:44.514715 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:44.515209 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:44.515216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:44.515220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:44.515590 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:44.515853 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:44.516345 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 02:42:44.516642 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 02:42:44.517768 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:44.496870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33733 (local address 127.17.196.193:53539)
0812 02:42:44.498374 (+ 1504us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:44.498401 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:44.498608 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:44.499005 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:44.499010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:44.499655 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:44.500396 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:44.500419 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:44.502109 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:44.502116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:44.504338 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:44.504369 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:44.505810 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:44.505821 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:44.506130 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:44.507285 (+ 1155us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:44.507325 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:44.509014 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:44.513788 (+ 4774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:44.513795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:44.513800 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:44.514351 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:44.514878 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:44.514893 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:44.514897 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:44.515063 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:44.515780 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:44.515786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:44.516041 (+ 255us) client_negotiation.cc:769] Sending connection context
0812 02:42:44.516226 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 02:42:44.516693 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"thread_start_us":791,"threads_started":1}
I20260812 02:42:44.526880 20200 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:41979
I20260812 02:42:44.526917 20346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:41979 every 8 connection(s)
I20260812 02:42:44.528360 20200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:42:44.531044 20347 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33733
I20260812 02:42:44.531294 20347 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:44.532006 20347 heartbeater.cc:511] Master 127.17.196.254:33733 requested a full tablet report, sending...
I20260812 02:42:44.534392 20120 ts_manager.cc:194] Registered new tserver with Master: 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979)
I20260812 02:42:44.536067 20120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:48469
I20260812 02:42:44.538246 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 20200
I20260812 02:42:44.538478 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:42:44.539183 18195 external_mini_cluster.cc:1428] Setting key e78698d309a0b03a0f295bb7dd43f0a9
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:42:44.594471 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33733
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:44.997702 20353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:44.998119 20353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:44.998328 20353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:45.006198 20353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:45.006417 20353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:45.006594 20353 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 02:42:45.006695 20353 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:42:45.006850 20353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:42:45.017632 20353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:33733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:45.019688 20353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:45.021318 20353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:45.036978 20358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:45.037065 20361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:45.037019 20359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:45.038237 20353 server_base.cc:1034] running on GCE node
I20260812 02:42:45.038972 20353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:45.040310 20353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:45.041626 20353 hybrid_clock.cc:648] HybridClock initialized: now 1786502565041565 us; error 36 us; skew 500 ppm
I20260812 02:42:45.042065 20353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:45 dist-test-slave-ss4n krb5kdc[20069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502565, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:45.045996 20353 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:42:45.047879 20353 webserver.cc:460] Webserver started at http://127.17.196.194:41341/ using document root <none> and password file <none>
I20260812 02:42:45.048794 20353 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:45.048925 20353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:45.049288 20353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:45.052675 20353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "fc0b4f88405f499da196fe6cfe918d5d"
format_stamp: "Formatted at 2026-08-12 02:42:45 on dist-test-slave-ss4n"
server_key: "0a51d3b42249659aae8634c6a4ff0bb6"
server_key_iv: "6af0db1bf2d14d5718dfebdfa7ba4962"
server_key_version: "encryptionkey@0"
I20260812 02:42:45.053743 20353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "fc0b4f88405f499da196fe6cfe918d5d"
format_stamp: "Formatted at 2026-08-12 02:42:45 on dist-test-slave-ss4n"
server_key: "0a51d3b42249659aae8634c6a4ff0bb6"
server_key_iv: "6af0db1bf2d14d5718dfebdfa7ba4962"
server_key_version: "encryptionkey@0"
I20260812 02:42:45.060417 20353 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260812 02:42:45.065089 20371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:45.066943 20353 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:42:45.067274 20353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "fc0b4f88405f499da196fe6cfe918d5d"
format_stamp: "Formatted at 2026-08-12 02:42:45 on dist-test-slave-ss4n"
server_key: "0a51d3b42249659aae8634c6a4ff0bb6"
server_key_iv: "6af0db1bf2d14d5718dfebdfa7ba4962"
server_key_version: "encryptionkey@0"
I20260812 02:42:45.067519 20353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:45.098603 20353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:45.103909 20353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:45.104287 20353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:45.107720 20353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:45.107865 20353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:45.107992 20353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:45.108172 20353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 12 02:42:45 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502565, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:45.139096 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.108383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:44821 (local address 127.17.196.254:33733)
0812 02:42:45.109327 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:45.109332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:45.110569 (+ 1237us) server_negotiation.cc:407] Connection header received
0812 02:42:45.113148 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:45.113155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:45.113256 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:45.113416 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:45.116055 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.117677 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.120722 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.121621 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.121813 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.128374 (+ 6561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:45.128386 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:45.128393 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:45.128453 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:45.133269 (+ 4816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.134823 (+ 1554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.134833 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.134839 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.135062 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.136037 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.136044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.136050 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.136518 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:45.136722 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:45.138188 (+ 1466us) server_negotiation.cc:299] Negotiation successful
0812 02:42:45.138540 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":705,"threads_started":1}
I20260812 02:42:45.141640 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.109176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33733 (local address 127.17.196.194:44821)
0812 02:42:45.110305 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:45.110356 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:45.112756 (+ 2400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:45.113760 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:45.113775 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:45.114724 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:45.115703 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.115729 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.117892 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.117901 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.120494 (+ 2593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.120520 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.121966 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.121977 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.122401 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.123462 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:45.123519 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:45.127990 (+ 4471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:45.133536 (+ 5546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:45.133557 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:45.133584 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:45.134628 (+ 1044us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:45.135246 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:45.135259 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:45.135267 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:45.135832 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:45.136730 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:45.136743 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:45.137555 (+ 812us) client_negotiation.cc:769] Sending connection context
0812 02:42:45.138059 (+ 504us) client_negotiation.cc:240] Negotiation successful
0812 02:42:45.138924 (+ 865us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":561,"threads_started":1}
I20260812 02:42:45.160449 20353 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:35093
I20260812 02:42:45.160612 20504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:35093 every 8 connection(s)
I20260812 02:42:45.162204 20353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:42:45.165105 20505 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33733
I20260812 02:42:45.165401 20505 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:45.166325 20505 heartbeater.cc:511] Master 127.17.196.254:33733 requested a full tablet report, sending...
I20260812 02:42:45.168175 20120 ts_manager.cc:194] Registered new tserver with Master: fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093)
I20260812 02:42:45.169672 20120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:44821
I20260812 02:42:45.171161 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.151357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:39051 (local address 127.17.196.254:33733)
0812 02:42:45.151619 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:45.151626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:45.152548 (+ 922us) server_negotiation.cc:407] Connection header received
0812 02:42:45.152775 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:45.152785 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:45.152907 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:45.153034 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:45.155401 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.156937 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.159339 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.160262 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.160429 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.163156 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:45.163167 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:45.163176 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:45.163239 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:45.167551 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.168654 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.168660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.168662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.168816 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.169303 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.169307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.169309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.169780 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:45.170025 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:45.170529 (+ 504us) server_negotiation.cc:299] Negotiation successful
0812 02:42:45.170789 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:42:45.171316 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.151502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33733 (local address 127.17.196.194:39051)
0812 02:42:45.152404 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:45.152427 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:45.152623 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:45.153122 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:45.153131 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:45.154264 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:45.155109 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.155136 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.157174 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.157184 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.159102 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.159130 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.160535 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.160543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.160945 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.161702 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:45.161723 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:45.162832 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:45.167741 (+ 4909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:45.167751 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:45.167758 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:45.168467 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:45.168969 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:45.168975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:45.168980 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:45.169165 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:45.169943 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:45.169951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:45.170184 (+ 233us) client_negotiation.cc:769] Sending connection context
0812 02:42:45.170350 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 02:42:45.170757 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":588,"threads_started":1}
I20260812 02:42:45.171455 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 20353
I20260812 02:42:45.171659 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:42:45.172308 18195 external_mini_cluster.cc:1428] Setting key 207bf99e08634fb084ac1eec8ed5219c
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:42:45.233793 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33733
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:42:45.539412 20347 heartbeater.cc:503] Master 127.17.196.254:33733 was elected leader, sending a full tablet report...
W20260812 02:42:45.606274 20509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:45.606608 20509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:45.606781 20509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:45.614639 20509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:45.614787 20509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:45.614928 20509 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 02:42:45.615020 20509 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:42:45.615142 20509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:42:45.625855 20509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:33733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:45.627341 20509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:45.628818 20509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:45.643292 20515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:45.643220 20519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:45.643891 20516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:45.644659 20509 server_base.cc:1034] running on GCE node
I20260812 02:42:45.645668 20509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:45.646983 20509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:45.648226 20509 hybrid_clock.cc:648] HybridClock initialized: now 1786502565648172 us; error 41 us; skew 500 ppm
I20260812 02:42:45.648658 20509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:45 dist-test-slave-ss4n krb5kdc[20069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502565, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:45.656407 20509 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:42:45.658628 20509 webserver.cc:460] Webserver started at http://127.17.196.195:38447/ using document root <none> and password file <none>
I20260812 02:42:45.659408 20509 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:45.659543 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:45.659854 20509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:45.663363 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "81a1cf7f8ee749abb2dfc0bd861452d6"
format_stamp: "Formatted at 2026-08-12 02:42:45 on dist-test-slave-ss4n"
server_key: "1c54a18e48a964245d4fc99614225883"
server_key_iv: "cb23ddbcbc05d6bdd9702fcc4c031a93"
server_key_version: "encryptionkey@0"
I20260812 02:42:45.664235 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "81a1cf7f8ee749abb2dfc0bd861452d6"
format_stamp: "Formatted at 2026-08-12 02:42:45 on dist-test-slave-ss4n"
server_key: "1c54a18e48a964245d4fc99614225883"
server_key_iv: "cb23ddbcbc05d6bdd9702fcc4c031a93"
server_key_version: "encryptionkey@0"
I20260812 02:42:45.670586 20509 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 02:42:45.675253 20526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:45.677182 20509 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:42:45.677441 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "81a1cf7f8ee749abb2dfc0bd861452d6"
format_stamp: "Formatted at 2026-08-12 02:42:45 on dist-test-slave-ss4n"
server_key: "1c54a18e48a964245d4fc99614225883"
server_key_iv: "cb23ddbcbc05d6bdd9702fcc4c031a93"
server_key_version: "encryptionkey@0"
I20260812 02:42:45.677798 20509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:45.703804 20509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:45.709197 20509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:45.709684 20509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:45.713052 20509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:45.713250 20509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:45.713418 20509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:45.713567 20509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:42:45 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502565, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:45.743921 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.713833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:42579 (local address 127.17.196.254:33733)
0812 02:42:45.714671 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:45.714679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:45.716041 (+ 1362us) server_negotiation.cc:407] Connection header received
0812 02:42:45.718757 (+ 2716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:45.718763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:45.718896 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:45.719034 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:45.723347 (+ 4313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.724691 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.728103 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.728892 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.729128 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.735057 (+ 5929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:45.735064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:45.735067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:45.735112 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:45.738672 (+ 3560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.740080 (+ 1408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.740089 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.740094 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.740337 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.741097 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.741102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.741104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.741425 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:45.741694 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:45.743016 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0812 02:42:45.743355 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":558,"threads_started":1}
I20260812 02:42:45.746143 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.714695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33733 (local address 127.17.196.195:42579)
0812 02:42:45.715804 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:45.715873 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:45.718354 (+ 2481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:45.719979 (+ 1625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:45.719995 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:45.721290 (+ 1295us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:45.723029 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.723072 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.724963 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.724970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.727877 (+ 2907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.727898 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.729283 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.729294 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.729783 (+ 489us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.731038 (+ 1255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:45.731097 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:45.734721 (+ 3624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:45.738904 (+ 4183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:45.738920 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:45.738952 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:45.739869 (+ 917us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:45.740486 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:45.740494 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:45.740501 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:45.740897 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:45.741673 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:45.741686 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:45.742363 (+ 677us) client_negotiation.cc:769] Sending connection context
0812 02:42:45.742849 (+ 486us) client_negotiation.cc:240] Negotiation successful
0812 02:42:45.743718 (+ 869us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":446,"threads_started":1}
I20260812 02:42:45.773556 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.753552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33733 (local address 127.17.196.195:37199)
0812 02:42:45.754525 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:45.754546 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:45.754767 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:45.755329 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:45.755338 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:45.756727 (+ 1389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:45.757362 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.757383 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.758588 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.758597 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.760839 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.760857 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.761780 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.761792 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.762042 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.762964 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:45.762998 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:45.764752 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:45.769295 (+ 4543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:45.769299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:45.769302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:45.769926 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:45.770518 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:45.770527 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:45.770534 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:45.770759 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:45.771813 (+ 1054us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:45.771822 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:45.772111 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 02:42:45.772315 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 02:42:45.772841 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":491,"threads_started":1}
I20260812 02:42:45.775029 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.753723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:37199 (local address 127.17.196.254:33733)
0812 02:42:45.753949 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:45.753961 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:45.754666 (+ 705us) server_negotiation.cc:407] Connection header received
0812 02:42:45.754972 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:45.754982 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:45.755115 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:45.755252 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:45.757614 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.758408 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.761036 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.761548 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.761700 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.765048 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:45.765057 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:45.765063 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:45.765115 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:45.769095 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.770101 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.770109 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.770115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.770326 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.770952 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.770960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.770966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.771472 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:45.771725 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:45.774124 (+ 2399us) server_negotiation.cc:299] Negotiation successful
0812 02:42:45.774499 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:42:45.780047 20509 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:40643
I20260812 02:42:45.780133 20656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:40643 every 8 connection(s)
I20260812 02:42:45.781694 20509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:42:45.784315 20657 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33733
I20260812 02:42:45.784650 20657 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:45.785372 20657 heartbeater.cc:511] Master 127.17.196.254:33733 requested a full tablet report, sending...
I20260812 02:42:45.787101 20119 ts_manager.cc:194] Registered new tserver with Master: 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:45.788127 20119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:42579
I20260812 02:42:45.791713 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 20509
I20260812 02:42:45.791975 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:42:45.792740 18195 external_mini_cluster.cc:1428] Setting key 367e8ba462834e0e7765e3bc3e0872a9
I20260812 02:42:45.796643 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:42:45 dist-test-slave-ss4n krb5kdc[20069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:42:45 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:45.857869 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.841455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.17.196.254:33733)
0812 02:42:45.841769 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:45.841776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:45.842211 (+ 435us) server_negotiation.cc:407] Connection header received
0812 02:42:45.842479 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:45.842484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:45.842561 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:45.842679 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:45.844473 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.845380 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.846674 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.847168 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.847370 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.851821 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:45.851826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:45.851829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:45.851869 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:45.854513 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.855241 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.855245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.855248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.855423 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:45.855998 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:45.856002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:45.856004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:45.856338 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:45.856618 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:45.857149 (+ 531us) server_negotiation.cc:299] Negotiation successful
0812 02:42:45.857359 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260812 02:42:45.863034 20119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:42:45.870755 20119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:45.911156 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.886121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.17.196.194:35093)
0812 02:42:45.901744 (+ 15623us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:45.901753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:45.901781 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:42:45.901893 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:45.901901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:45.902155 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:45.902396 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:45.903528 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.904999 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.908362 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.909792 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.910030 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.910243 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:45.910622 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 02:42:45.910813 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15458,"thread_start_us":15275,"threads_started":1}
I20260812 02:42:45.911502 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.887792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.17.196.193:41979)
0812 02:42:45.900412 (+ 12620us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:45.900427 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:45.902643 (+ 2216us) server_negotiation.cc:407] Connection header received
0812 02:42:45.902765 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:45.902774 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:45.903024 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:45.903247 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:45.903996 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.905734 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.907995 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.909871 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.910075 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.910288 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:45.910996 (+ 708us) server_negotiation.cc:299] Negotiation successful
0812 02:42:45.911176 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12335,"thread_start_us":12644,"threads_started":1}
I20260812 02:42:45.911450 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.885658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40643 (local address 127.0.0.1:45468)
0812 02:42:45.886698 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:45.886788 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:45.887056 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:45.902404 (+ 15348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:45.902409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:45.902487 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:45.903212 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.903235 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.905712 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.905718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.907999 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.908011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.910215 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.910224 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.910442 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.910511 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 02:42:45.910605 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 02:42:45.910852 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":90,"spinlock_wait_cycles":39040,"thread_start_us":671,"threads_started":1}
I20260812 02:42:45.911983 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.885749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35093 (local address 127.0.0.1:60110)
0812 02:42:45.896154 (+ 10405us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:45.896186 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:45.896424 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:45.902400 (+ 5976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:45.902409 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:45.902461 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:45.903294 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.903312 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.905153 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.905161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.907869 (+ 2708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.908172 (+ 303us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.910054 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.910061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.910329 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.910378 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 02:42:45.910549 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 02:42:45.910827 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10131,"mutex_wait_us":81,"spinlock_wait_cycles":4480,"thread_start_us":16238,"threads_started":1}
I20260812 02:42:45.912451 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.886202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.17.196.195:40643)
0812 02:42:45.901430 (+ 15228us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:45.901442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:45.901528 (+ 86us) server_negotiation.cc:407] Connection header received
0812 02:42:45.901666 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:45.901676 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:45.902018 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:45.902773 (+ 755us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:45.903517 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.905565 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.908196 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.909871 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.911497 (+ 1626us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.911758 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:45.911841 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 02:42:45.912062 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15002,"thread_start_us":14801,"threads_started":1}
I20260812 02:42:45.912505 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:45.887433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:41979 (local address 127.0.0.1:44408)
0812 02:42:45.902305 (+ 14872us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:45.902327 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:45.902478 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:45.903260 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:45.903273 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:45.903313 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:45.903754 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.903767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.905914 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.905924 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.907790 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:45.907807 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:45.910446 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:45.910452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:45.910704 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:45.910736 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 02:42:45.910819 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 02:42:45.910945 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14721,"mutex_wait_us":8,"spinlock_wait_cycles":11520,"thread_start_us":8223,"threads_started":1}
I20260812 02:42:45.916350 20427 tablet_service.cc:1467] Processing CreateTablet for tablet 16697f13e8874dab83c4da8a5cc23edc (DEFAULT_TABLE table=test-table [id=796a011740204a77a8788d142f268cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:45.916571 20272 tablet_service.cc:1467] Processing CreateTablet for tablet 16697f13e8874dab83c4da8a5cc23edc (DEFAULT_TABLE table=test-table [id=796a011740204a77a8788d142f268cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:45.918238 20427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16697f13e8874dab83c4da8a5cc23edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:45.918313 20272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16697f13e8874dab83c4da8a5cc23edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:45.919941 20580 tablet_service.cc:1467] Processing CreateTablet for tablet 16697f13e8874dab83c4da8a5cc23edc (DEFAULT_TABLE table=test-table [id=796a011740204a77a8788d142f268cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:45.922485 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16697f13e8874dab83c4da8a5cc23edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:45.934019 20678 tablet_bootstrap.cc:492] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Bootstrap starting.
I20260812 02:42:45.936424 20679 tablet_bootstrap.cc:492] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Bootstrap starting.
I20260812 02:42:45.939190 20678 tablet_bootstrap.cc:654] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:45.940373 20680 tablet_bootstrap.cc:492] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Bootstrap starting.
I20260812 02:42:45.940634 20679 tablet_bootstrap.cc:654] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:45.940805 20678 log.cc:824] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:45.942245 20679 log.cc:824] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:45.944707 20680 tablet_bootstrap.cc:654] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:45.944944 20678 tablet_bootstrap.cc:492] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: No bootstrap required, opened a new log
I20260812 02:42:45.945289 20678 ts_tablet_manager.cc:1397] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260812 02:42:45.946007 20679 tablet_bootstrap.cc:492] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: No bootstrap required, opened a new log
I20260812 02:42:45.946141 20680 log.cc:824] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:45.946307 20679 ts_tablet_manager.cc:1397] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260812 02:42:45.956193 20680 tablet_bootstrap.cc:492] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: No bootstrap required, opened a new log
I20260812 02:42:45.956629 20680 ts_tablet_manager.cc:1397] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260812 02:42:45.958000 20678 raft_consensus.cc:348] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:45.958714 20678 raft_consensus.cc:374] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:45.958434 20679 raft_consensus.cc:348] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:45.958884 20678 raft_consensus.cc:729] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbc2147296446e58d6ad07c2851d0a4, State: Initialized, Role: FOLLOWER
I20260812 02:42:45.959024 20679 raft_consensus.cc:374] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:45.959139 20679 raft_consensus.cc:729] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0b4f88405f499da196fe6cfe918d5d, State: Initialized, Role: FOLLOWER
I20260812 02:42:45.959666 20678 consensus_queue.cc:260] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:45.959870 20679 consensus_queue.cc:260] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:45.964046 20678 ts_tablet_manager.cc:1428] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Time spent starting tablet: real 0.019s user 0.010s sys 0.006s
I20260812 02:42:45.965297 20505 heartbeater.cc:503] Master 127.17.196.254:33733 was elected leader, sending a full tablet report...
I20260812 02:42:45.966801 20679 ts_tablet_manager.cc:1428] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
I20260812 02:42:45.967178 20680 raft_consensus.cc:348] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:45.967687 20680 raft_consensus.cc:374] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:45.967844 20680 raft_consensus.cc:729] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a1cf7f8ee749abb2dfc0bd861452d6, State: Initialized, Role: FOLLOWER
I20260812 02:42:45.968592 20680 consensus_queue.cc:260] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:45.969983 20657 heartbeater.cc:503] Master 127.17.196.254:33733 was elected leader, sending a full tablet report...
I20260812 02:42:45.970567 20680 ts_tablet_manager.cc:1428] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
W20260812 02:42:46.031307 20348 tablet.cc:2369] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:42:46.033584 20658 tablet.cc:2369] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:42:46.165047 20506 tablet.cc:2369] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:42:46.220793 20685 raft_consensus.cc:482] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:46.221220 20685 raft_consensus.cc:504] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:46.223126 20685 leader_election.cc:288] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643), 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979)
I20260812 02:42:46.232146 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.224674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:34249 (local address 127.17.196.193:41979)
0812 02:42:46.224942 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.224958 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.225213 (+ 255us) server_negotiation.cc:407] Connection header received
0812 02:42:46.225383 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.225388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.225473 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.225660 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:46.226234 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.227189 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.229639 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.230904 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.231086 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.231163 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.231766 (+ 603us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.231896 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 02:42:46.232168 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.223758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:60447 (local address 127.17.196.195:40643)
0812 02:42:46.223996 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.224001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.224445 (+ 444us) server_negotiation.cc:407] Connection header received
0812 02:42:46.224626 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.224633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.224744 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.224868 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:46.225735 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.226810 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.229639 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.230904 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.231086 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.231163 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.231758 (+ 595us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.231896 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:42:46.232502 20688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.224378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:41979 (local address 127.17.196.194:34249)
0812 02:42:46.225069 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:46.225086 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:46.225246 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:46.225676 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:46.225682 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:46.225715 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:46.226016 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.226024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.227316 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.227321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.229409 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.229424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.231215 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.231223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.231527 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.231607 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 02:42:46.231708 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 02:42:46.231890 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":107,"thread_start_us":391,"threads_started":1}
I20260812 02:42:46.232717 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.223484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40643 (local address 127.17.196.194:60447)
0812 02:42:46.224298 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:46.224324 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:46.224482 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:46.225059 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:46.225064 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:46.225092 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:46.225545 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.225564 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.226953 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.226956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.229320 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.229337 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.231231 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.231236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.231528 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.231611 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 02:42:46.231708 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 02:42:46.231890 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":107,"thread_start_us":439,"threads_started":1}
I20260812 02:42:46.233726 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16697f13e8874dab83c4da8a5cc23edc" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbc2147296446e58d6ad07c2851d0a4" is_pre_election: true
I20260812 02:42:46.233778 20601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16697f13e8874dab83c4da8a5cc23edc" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" is_pre_election: true
I20260812 02:42:46.234264 20299 raft_consensus.cc:2454] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 0.
I20260812 02:42:46.234266 20601 raft_consensus.cc:2454] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 0.
I20260812 02:42:46.234931 20375 leader_election.cc:302] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81a1cf7f8ee749abb2dfc0bd861452d6, fc0b4f88405f499da196fe6cfe918d5d; no voters:
I20260812 02:42:46.235360 20685 raft_consensus.cc:2781] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:46.235562 20685 raft_consensus.cc:482] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:46.235736 20685 raft_consensus.cc:3037] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:46.238552 20685 raft_consensus.cc:504] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:46.239238 20685 leader_election.cc:288] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 election: Requested vote from peers 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643), 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979)
I20260812 02:42:46.240060 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16697f13e8874dab83c4da8a5cc23edc" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbc2147296446e58d6ad07c2851d0a4"
I20260812 02:42:46.240067 20601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16697f13e8874dab83c4da8a5cc23edc" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6"
I20260812 02:42:46.240314 20299 raft_consensus.cc:3037] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:46.240330 20601 raft_consensus.cc:3037] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:46.243024 20299 raft_consensus.cc:2454] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 1.
I20260812 02:42:46.243103 20601 raft_consensus.cc:2454] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 1.
I20260812 02:42:46.243623 20375 leader_election.cc:302] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, fc0b4f88405f499da196fe6cfe918d5d; no voters:
I20260812 02:42:46.244167 20685 raft_consensus.cc:2781] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:46.244836 20685 raft_consensus.cc:686] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 1 LEADER]: Becoming Leader. State: Replica: fc0b4f88405f499da196fe6cfe918d5d, State: Running, Role: LEADER
I20260812 02:42:46.245420 20685 consensus_queue.cc:237] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } }
I20260812 02:42:46.251555 20120 catalog_manager.cc:5495] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d reported cstate change: term changed from 0 to 1, leader changed from <none> to fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194). New cstate: current_term: 1 leader_uuid: "fc0b4f88405f499da196fe6cfe918d5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } health_report { overall_health: UNKNOWN } } }
Aug 12 02:42:46 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502563, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:46.316228 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.300560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33733 (local address 127.0.0.1:39504)
0812 02:42:46.301206 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:46.301223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:46.301408 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:46.301867 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:46.301872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:46.302682 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:46.302998 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.303010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.304169 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.304177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.305249 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.305261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.306209 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.306216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.306385 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.307140 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:46.307161 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:46.309803 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:46.313197 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:46.313213 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:46.313220 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:46.313800 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:46.314235 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:46.314240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:46.314243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:46.314538 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:46.315167 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:46.315175 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:46.315362 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 02:42:46.315595 (+ 233us) client_negotiation.cc:240] Negotiation successful
0812 02:42:46.315845 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":346,"threads_started":1}
I20260812 02:42:46.316272 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.300459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.17.196.254:33733)
0812 02:42:46.300724 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.300730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.301321 (+ 591us) server_negotiation.cc:407] Connection header received
0812 02:42:46.301624 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.301630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.301705 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.301825 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:46.303181 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.304014 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.305413 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.306004 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.306172 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.310079 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:46.310087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:46.310093 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:46.310145 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:46.313020 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:46.313928 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:46.313932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:46.313935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:46.314087 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:46.314682 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:46.314686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:46.314688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:46.315012 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:46.315130 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.315717 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.316021 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":136}
I20260812 02:42:46.322376 20120 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:39504:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 02:42:46.323558 20120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:46.331185 20427 tablet_service.cc:1467] Processing CreateTablet for tablet 54508925f68f4943954915d886925e91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=057c7e2be7fe4bff912bf94f0a3ca001]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:42:46.331040 20272 tablet_service.cc:1467] Processing CreateTablet for tablet 54508925f68f4943954915d886925e91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=057c7e2be7fe4bff912bf94f0a3ca001]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:42:46.331398 20580 tablet_service.cc:1467] Processing CreateTablet for tablet 54508925f68f4943954915d886925e91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=057c7e2be7fe4bff912bf94f0a3ca001]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:42:46.332072 20427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54508925f68f4943954915d886925e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:46.332060 20272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54508925f68f4943954915d886925e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:46.332161 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54508925f68f4943954915d886925e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:46.338577 20679 tablet_bootstrap.cc:492] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: Bootstrap starting.
I20260812 02:42:46.339376 20680 tablet_bootstrap.cc:492] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: Bootstrap starting.
I20260812 02:42:46.340220 20678 tablet_bootstrap.cc:492] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: Bootstrap starting.
I20260812 02:42:46.340402 20679 tablet_bootstrap.cc:654] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:46.341130 20680 tablet_bootstrap.cc:654] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:46.341439 20678 tablet_bootstrap.cc:654] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:46.343670 20680 tablet_bootstrap.cc:492] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: No bootstrap required, opened a new log
I20260812 02:42:46.343832 20680 ts_tablet_manager.cc:1397] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 02:42:46.344233 20679 tablet_bootstrap.cc:492] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: No bootstrap required, opened a new log
I20260812 02:42:46.344451 20679 ts_tablet_manager.cc:1397] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 02:42:46.344671 20678 tablet_bootstrap.cc:492] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: No bootstrap required, opened a new log
I20260812 02:42:46.344681 20680 raft_consensus.cc:348] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.344841 20678 ts_tablet_manager.cc:1397] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 02:42:46.344957 20680 raft_consensus.cc:374] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:46.345072 20680 raft_consensus.cc:729] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a1cf7f8ee749abb2dfc0bd861452d6, State: Initialized, Role: FOLLOWER
I20260812 02:42:46.345319 20680 consensus_queue.cc:260] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.345355 20679 raft_consensus.cc:348] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.345664 20679 raft_consensus.cc:374] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:46.345767 20679 raft_consensus.cc:729] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0b4f88405f499da196fe6cfe918d5d, State: Initialized, Role: FOLLOWER
I20260812 02:42:46.345736 20678 raft_consensus.cc:348] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.345985 20678 raft_consensus.cc:374] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:46.346001 20680 ts_tablet_manager.cc:1428] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:42:46.346086 20678 raft_consensus.cc:729] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbc2147296446e58d6ad07c2851d0a4, State: Initialized, Role: FOLLOWER
I20260812 02:42:46.346045 20679 consensus_queue.cc:260] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.346320 20678 consensus_queue.cc:260] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.346761 20679 ts_tablet_manager.cc:1428] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:42:46.346602 20685 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.346897 20678 ts_tablet_manager.cc:1428] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:42:46.346969 20685 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:46.347458 20686 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.347802 20686 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:46.347894 20684 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.348300 20684 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:46.581642 20686 raft_consensus.cc:482] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:46.581943 20686 raft_consensus.cc:504] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.583878 20686 leader_election.cc:288] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979), fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093)
I20260812 02:42:46.591051 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.584394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:42563 (local address 127.17.196.193:41979)
0812 02:42:46.584790 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.584800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.585126 (+ 326us) server_negotiation.cc:407] Connection header received
0812 02:42:46.585539 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.585545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.585649 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.585757 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:46.586431 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.587383 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.589224 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.590087 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.590290 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.590372 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.590636 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.590771 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260812 02:42:46.591224 20700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.584038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:41979 (local address 127.17.196.195:42563)
0812 02:42:46.584981 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:46.585010 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:46.585302 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:46.585837 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:46.585844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:46.585875 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:46.586239 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.586262 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.587552 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.587558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.589040 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.589054 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.590279 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.590285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.590462 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.590488 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 02:42:46.590603 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 02:42:46.590746 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":52,"thread_start_us":583,"threads_started":1}
I20260812 02:42:46.591952 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.584453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:49399 (local address 127.17.196.194:35093)
0812 02:42:46.585217 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.585226 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.585797 (+ 571us) server_negotiation.cc:407] Connection header received
0812 02:42:46.585883 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.585888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.585965 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.586086 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:46.586692 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.587425 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.589664 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.590583 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.590803 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.590882 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.591389 (+ 507us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.591571 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":510,"threads_started":1}
I20260812 02:42:46.591965 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.584324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35093 (local address 127.17.196.195:49399)
0812 02:42:46.585590 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:46.585610 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:46.585756 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:46.586112 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:46.586116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:46.586140 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:46.586504 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.586518 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.587604 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.587612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.589432 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.589446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.590833 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.590840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.591042 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.591068 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 02:42:46.591413 (+ 345us) client_negotiation.cc:240] Negotiation successful
0812 02:42:46.591562 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1113,"mutex_wait_us":46,"thread_start_us":917,"threads_started":1}
I20260812 02:42:46.591979 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54508925f68f4943954915d886925e91" candidate_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbc2147296446e58d6ad07c2851d0a4" is_pre_election: true
I20260812 02:42:46.592303 20299 raft_consensus.cc:2454] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81a1cf7f8ee749abb2dfc0bd861452d6 in term 0.
I20260812 02:42:46.592941 20530 leader_election.cc:302] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, 81a1cf7f8ee749abb2dfc0bd861452d6; no voters:
I20260812 02:42:46.593465 20686 raft_consensus.cc:2781] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:46.593292 20448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54508925f68f4943954915d886925e91" candidate_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0b4f88405f499da196fe6cfe918d5d" is_pre_election: true
I20260812 02:42:46.593677 20686 raft_consensus.cc:482] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:46.593813 20686 raft_consensus.cc:3037] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:46.593842 20448 raft_consensus.cc:2454] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81a1cf7f8ee749abb2dfc0bd861452d6 in term 0.
I20260812 02:42:46.596570 20686 raft_consensus.cc:504] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.597211 20686 leader_election.cc:288] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [CANDIDATE]: Term 1 election: Requested vote from peers 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979), fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093)
I20260812 02:42:46.597997 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54508925f68f4943954915d886925e91" candidate_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbc2147296446e58d6ad07c2851d0a4"
I20260812 02:42:46.598042 20448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54508925f68f4943954915d886925e91" candidate_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0b4f88405f499da196fe6cfe918d5d"
I20260812 02:42:46.598265 20448 raft_consensus.cc:3037] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:46.598299 20299 raft_consensus.cc:3037] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:46.601819 20299 raft_consensus.cc:2454] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81a1cf7f8ee749abb2dfc0bd861452d6 in term 1.
I20260812 02:42:46.601835 20448 raft_consensus.cc:2454] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81a1cf7f8ee749abb2dfc0bd861452d6 in term 1.
I20260812 02:42:46.602347 20530 leader_election.cc:302] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, 81a1cf7f8ee749abb2dfc0bd861452d6; no voters:
I20260812 02:42:46.602748 20686 raft_consensus.cc:2781] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:46.603260 20686 raft_consensus.cc:686] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 LEADER]: Becoming Leader. State: Replica: 81a1cf7f8ee749abb2dfc0bd861452d6, State: Running, Role: LEADER
I20260812 02:42:46.603819 20686 consensus_queue.cc:237] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.607686 20703 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: TxnStatusTablet state changed. Reason: New leader 81a1cf7f8ee749abb2dfc0bd861452d6. Latest consensus state: current_term: 1 leader_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.608162 20703 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:42:46.608992 20705 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 02:42:46.609329 20705 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 02:42:46.609310 20118 catalog_manager.cc:5495] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } health_report { overall_health: HEALTHY } } }
I20260812 02:42:46.655601 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.648064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45472 (local address 127.17.196.195:40643)
0812 02:42:46.648232 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.648237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.648255 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:42:46.648614 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.648621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.648781 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.648922 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:46.649588 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.650650 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.652918 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.653410 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.653620 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.654262 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:46.654893 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:46.655019 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.655218 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.655298 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:42:46.655732 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.647726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40643 (local address 127.0.0.1:45472)
0812 02:42:46.648061 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:46.648086 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:46.648250 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:46.649004 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:46.649010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:46.649025 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:42:46.649261 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.649274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.650809 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.650815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.652695 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.652718 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.653698 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.653705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.653948 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.654038 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:42:46.655033 (+ 995us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:42:46.655071 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 02:42:46.655183 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 02:42:46.655379 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 02:42:46.664904 20299 raft_consensus.cc:1261] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Refusing update from remote peer 81a1cf7f8ee749abb2dfc0bd861452d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:46.664867 20448 raft_consensus.cc:1261] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Refusing update from remote peer 81a1cf7f8ee749abb2dfc0bd861452d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:46.665953 20703 consensus_queue.cc:1035] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:46.666308 20686 consensus_queue.cc:1035] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:46.681516 20685 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: TxnStatusTablet state changed. Reason: New leader 81a1cf7f8ee749abb2dfc0bd861452d6. Latest consensus state: current_term: 1 leader_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.681937 20685 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:46.685355 20708 mvcc.cc:205] Tried to move back new op lower bound from 7317514513048326144 to 7317514512813793280. Current Snapshot: MvccSnapshot[applied={T|T < 7317514513048326144}]
I20260812 02:42:46.686055 20684 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: TxnStatusTablet state changed. Reason: New leader 81a1cf7f8ee749abb2dfc0bd861452d6. Latest consensus state: current_term: 1 leader_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.686414 20684 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:46.692786 20684 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.693221 20684 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P 1dbc2147296446e58d6ad07c2851d0a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:46.695353 20703 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.695711 20703 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:42:46.695981 20703 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.696233 20703 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P 81a1cf7f8ee749abb2dfc0bd861452d6: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:42:46.701999 20685 tablet_replica.cc:439] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } } }
I20260812 02:42:46.702361 20685 tablet_replica.cc:442] T 54508925f68f4943954915d886925e91 P fc0b4f88405f499da196fe6cfe918d5d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:42:46.702651 20711 mvcc.cc:205] Tried to move back new op lower bound from 7317514513048326144 to 7317514512813793280. Current Snapshot: MvccSnapshot[applied={T|T < 7317514513048326144 or (T in {7317514513048326144})}]
I20260812 02:42:46.711457 20685 consensus_queue.cc:1035] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:46.716977 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.707893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.17.196.194:35093)
0812 02:42:46.708169 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.708179 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.708784 (+ 605us) server_negotiation.cc:407] Connection header received
0812 02:42:46.709065 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.709074 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.709211 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.709360 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:46.710185 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.711344 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.713521 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.714218 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.714305 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.715077 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:46.716110 (+ 1033us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:46.716249 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.716576 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.716675 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:42:46.719614 20689 consensus_queue.cc:1035] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:46.739202 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.732022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40643 (local address 127.17.196.194:46809)
0812 02:42:46.732729 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:46.732745 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:46.732977 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:46.733464 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:46.733468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:46.733560 (+ 92us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:42:46.733726 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.733735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.735195 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.735201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.736480 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.736498 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.737728 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.737735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.737922 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.738004 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:42:46.738565 (+ 561us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:42:46.738582 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 02:42:46.738680 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 02:42:46.738839 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":435,"threads_started":1}
I20260812 02:42:46.739179 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.732136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:46809 (local address 127.17.196.195:40643)
0812 02:42:46.732330 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.732336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.732860 (+ 524us) server_negotiation.cc:407] Connection header received
0812 02:42:46.733226 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.733233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.733329 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.733437 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:46.733868 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.735014 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.736650 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.737367 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.737544 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.738302 (+ 758us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:46.738436 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:46.738560 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.738727 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.738821 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 02:42:46.810932 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.778578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:39243 (local address 127.17.196.194:35093)
0812 02:42:46.778787 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:46.778796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:46.800678 (+ 21882us) server_negotiation.cc:407] Connection header received
0812 02:42:46.801030 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:46.801038 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:46.801135 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:46.801199 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:46.802203 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.803475 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.808407 (+ 4932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.809104 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.809214 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.810071 (+ 857us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:46.810227 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:46.810306 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:46.810517 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 02:42:46.810617 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:42:46.814865 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:46.778633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35093 (local address 127.17.196.195:39243)
0812 02:42:46.800536 (+ 21903us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:46.800565 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:46.800825 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:46.801425 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:46.801430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:46.801443 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:42:46.801958 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.801971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.803685 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.803691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.808203 (+ 4512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:46.808223 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:46.809444 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:46.809451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:46.809725 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:46.809812 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:42:46.810374 (+ 562us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:42:46.810384 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 02:42:46.814140 (+ 3756us) client_negotiation.cc:240] Negotiation successful
0812 02:42:46.814349 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21634,"thread_start_us":19082,"threads_started":1}
I20260812 02:42:46.873356 20119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:46.873775 20119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:46.879068 20119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: Sending DeleteTablet for 3 replicas of tablet 16697f13e8874dab83c4da8a5cc23edc
I20260812 02:42:46.880460 20427 tablet_service.cc:1514] Processing DeleteTablet for tablet 16697f13e8874dab83c4da8a5cc23edc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:46 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:60110
I20260812 02:42:46.881273 20580 tablet_service.cc:1514] Processing DeleteTablet for tablet 16697f13e8874dab83c4da8a5cc23edc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:46 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:45468
I20260812 02:42:46.882117 20272 tablet_service.cc:1514] Processing DeleteTablet for tablet 16697f13e8874dab83c4da8a5cc23edc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:46 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:44408
I20260812 02:42:46.882599 20735 tablet_replica.cc:330] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: stopping tablet replica
I20260812 02:42:46.883209 20735 raft_consensus.cc:2229] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:46.883739 20735 raft_consensus.cc:2258] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:46.884179 20120 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:42:46.885075 20120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:46.886633 20735 ts_tablet_manager.cc:1905] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:46.887898 20734 tablet_replica.cc:330] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: stopping tablet replica
I20260812 02:42:46.888376 20734 raft_consensus.cc:2229] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:42:46.889120 20734 raft_consensus.cc:2258] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:46.895948 20735 ts_tablet_manager.cc:1918] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:46.896019 20580 tablet_service.cc:1467] Processing CreateTablet for tablet d408db78af7d4a16ab65417f0841ff50 (DEFAULT_TABLE table=test-table [id=06544b6fd3cd4b63afafb00fafeb1cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:46.896382 20735 log.cc:1196] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/16697f13e8874dab83c4da8a5cc23edc
I20260812 02:42:46.896337 20271 tablet_service.cc:1467] Processing CreateTablet for tablet d408db78af7d4a16ab65417f0841ff50 (DEFAULT_TABLE table=test-table [id=06544b6fd3cd4b63afafb00fafeb1cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:46.896557 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d408db78af7d4a16ab65417f0841ff50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:46.896813 20271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d408db78af7d4a16ab65417f0841ff50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:46.897394 20735 ts_tablet_manager.cc:1939] T 16697f13e8874dab83c4da8a5cc23edc P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting consensus metadata
I20260812 02:42:46.899808 20106 catalog_manager.cc:4841] TS 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643): tablet 16697f13e8874dab83c4da8a5cc23edc (table test-table [id=796a011740204a77a8788d142f268cf1]) successfully deleted
I20260812 02:42:46.901144 20736 tablet_replica.cc:330] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: stopping tablet replica
I20260812 02:42:46.901751 20736 raft_consensus.cc:2229] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:46.901995 20427 tablet_service.cc:1467] Processing CreateTablet for tablet d408db78af7d4a16ab65417f0841ff50 (DEFAULT_TABLE table=test-table [id=06544b6fd3cd4b63afafb00fafeb1cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:46.902212 20736 raft_consensus.cc:2258] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:46.902591 20427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d408db78af7d4a16ab65417f0841ff50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:46.904372 20734 ts_tablet_manager.cc:1905] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:46.913811 20734 ts_tablet_manager.cc:1918] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:46.914150 20734 log.cc:1196] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/16697f13e8874dab83c4da8a5cc23edc
I20260812 02:42:46.914909 20734 ts_tablet_manager.cc:1939] T 16697f13e8874dab83c4da8a5cc23edc P fc0b4f88405f499da196fe6cfe918d5d: Deleting consensus metadata
I20260812 02:42:46.917058 20737 tablet_bootstrap.cc:492] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: Bootstrap starting.
I20260812 02:42:46.917446 20105 catalog_manager.cc:4841] TS fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093): tablet 16697f13e8874dab83c4da8a5cc23edc (table test-table [id=796a011740204a77a8788d142f268cf1]) successfully deleted
I20260812 02:42:46.919378 20737 tablet_bootstrap.cc:654] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:46.919376 20736 ts_tablet_manager.cc:1905] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:46.923648 20737 tablet_bootstrap.cc:492] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: No bootstrap required, opened a new log
I20260812 02:42:46.924006 20737 ts_tablet_manager.cc:1397] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 02:42:46.925145 20737 raft_consensus.cc:348] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.925765 20737 raft_consensus.cc:374] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:46.925980 20737 raft_consensus.cc:729] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a1cf7f8ee749abb2dfc0bd861452d6, State: Initialized, Role: FOLLOWER
I20260812 02:42:46.926429 20737 consensus_queue.cc:260] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.927582 20737 ts_tablet_manager.cc:1428] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 02:42:46.939281 20738 tablet_bootstrap.cc:492] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: Bootstrap starting.
I20260812 02:42:46.940248 20740 tablet_bootstrap.cc:492] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: Bootstrap starting.
I20260812 02:42:46.942574 20740 tablet_bootstrap.cc:654] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:46.942624 20738 tablet_bootstrap.cc:654] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:46.943823 20736 ts_tablet_manager.cc:1918] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:46.944171 20736 log.cc:1196] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/16697f13e8874dab83c4da8a5cc23edc
I20260812 02:42:46.945133 20736 ts_tablet_manager.cc:1939] T 16697f13e8874dab83c4da8a5cc23edc P 1dbc2147296446e58d6ad07c2851d0a4: Deleting consensus metadata
I20260812 02:42:46.947815 20106 catalog_manager.cc:4841] TS 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979): tablet 16697f13e8874dab83c4da8a5cc23edc (table test-table [id=796a011740204a77a8788d142f268cf1]) successfully deleted
I20260812 02:42:46.950204 20740 tablet_bootstrap.cc:492] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: No bootstrap required, opened a new log
I20260812 02:42:46.950301 20738 tablet_bootstrap.cc:492] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: No bootstrap required, opened a new log
I20260812 02:42:46.950496 20738 ts_tablet_manager.cc:1397] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260812 02:42:46.950518 20740 ts_tablet_manager.cc:1397] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260812 02:42:46.951442 20740 raft_consensus.cc:348] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.951458 20738 raft_consensus.cc:348] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.951745 20738 raft_consensus.cc:374] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:46.951861 20738 raft_consensus.cc:729] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0b4f88405f499da196fe6cfe918d5d, State: Initialized, Role: FOLLOWER
I20260812 02:42:46.952119 20740 raft_consensus.cc:374] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:46.952313 20740 raft_consensus.cc:729] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbc2147296446e58d6ad07c2851d0a4, State: Initialized, Role: FOLLOWER
I20260812 02:42:46.952220 20738 consensus_queue.cc:260] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.952741 20740 consensus_queue.cc:260] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:46.953584 20740 ts_tablet_manager.cc:1428] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 02:42:46.954370 20738 ts_tablet_manager.cc:1428] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 02:42:47.003286 20684 raft_consensus.cc:482] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:47.003571 20684 raft_consensus.cc:504] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:47.005648 20684 leader_election.cc:288] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093), 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:47.024994 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.006725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:42275 (local address 127.17.196.195:40643)
0812 02:42:47.006918 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:47.006926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:47.016292 (+ 9366us) server_negotiation.cc:407] Connection header received
0812 02:42:47.016971 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:47.016981 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:47.017089 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:47.017188 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:47.018193 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.019402 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.022093 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.023596 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.023728 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.023817 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:47.024415 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 02:42:47.024579 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:42:47.025861 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.007118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:44609 (local address 127.17.196.194:35093)
0812 02:42:47.007382 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:47.007391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:47.017971 (+ 10580us) server_negotiation.cc:407] Connection header received
0812 02:42:47.018430 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:47.018438 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:47.018524 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:47.018595 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:42:47.019386 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.020513 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.023076 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.024422 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.024581 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.024680 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:47.025354 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 02:42:47.025528 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:42:47.026777 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.006228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35093 (local address 127.17.196.193:44609)
0812 02:42:47.017832 (+ 11604us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:47.017861 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:47.018237 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:47.018806 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:47.018811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:47.018835 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:47.019191 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:47.019202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.020718 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.020723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:47.022881 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:47.022896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.024904 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.024910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:47.025098 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.025136 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 02:42:47.026170 (+ 1034us) client_negotiation.cc:240] Negotiation successful
0812 02:42:47.026317 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11464,"mutex_wait_us":27,"thread_start_us":11759,"threads_started":1}
I20260812 02:42:47.027478 20448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d408db78af7d4a16ab65417f0841ff50" candidate_uuid: "1dbc2147296446e58d6ad07c2851d0a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0b4f88405f499da196fe6cfe918d5d" is_pre_election: true
I20260812 02:42:47.027791 20448 raft_consensus.cc:2454] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbc2147296446e58d6ad07c2851d0a4 in term 0.
I20260812 02:42:47.028353 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.005962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40643 (local address 127.17.196.193:42275)
0812 02:42:47.016079 (+ 10117us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:47.016129 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:47.016734 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:47.017464 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:47.017472 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:47.017533 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:42:47.017968 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:47.017983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.019589 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.019594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:47.021895 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:47.021910 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.023998 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.024006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:47.024203 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.024239 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 02:42:47.027605 (+ 3366us) client_negotiation.cc:240] Negotiation successful
0812 02:42:47.027856 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9908,"mutex_wait_us":1,"thread_start_us":10367,"threads_started":1}
I20260812 02:42:47.028398 20218 leader_election.cc:302] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, fc0b4f88405f499da196fe6cfe918d5d; no voters:
I20260812 02:42:47.028885 20684 raft_consensus.cc:2781] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:47.029021 20684 raft_consensus.cc:482] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:47.029122 20684 raft_consensus.cc:3037] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:47.029755 20601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d408db78af7d4a16ab65417f0841ff50" candidate_uuid: "1dbc2147296446e58d6ad07c2851d0a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" is_pre_election: true
I20260812 02:42:47.030041 20601 raft_consensus.cc:2454] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbc2147296446e58d6ad07c2851d0a4 in term 0.
I20260812 02:42:47.031951 20684 raft_consensus.cc:504] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:47.032779 20684 leader_election.cc:288] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [CANDIDATE]: Term 1 election: Requested vote from peers fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093), 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:47.033447 20448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d408db78af7d4a16ab65417f0841ff50" candidate_uuid: "1dbc2147296446e58d6ad07c2851d0a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0b4f88405f499da196fe6cfe918d5d"
I20260812 02:42:47.033764 20448 raft_consensus.cc:3037] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:47.036437 20448 raft_consensus.cc:2454] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dbc2147296446e58d6ad07c2851d0a4 in term 1.
I20260812 02:42:47.036887 20601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d408db78af7d4a16ab65417f0841ff50" candidate_uuid: "1dbc2147296446e58d6ad07c2851d0a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6"
I20260812 02:42:47.036975 20218 leader_election.cc:302] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, fc0b4f88405f499da196fe6cfe918d5d; no voters:
I20260812 02:42:47.037173 20601 raft_consensus.cc:3037] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:47.037451 20684 raft_consensus.cc:2781] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:47.041646 20601 raft_consensus.cc:2454] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dbc2147296446e58d6ad07c2851d0a4 in term 1.
I20260812 02:42:47.042948 20684 raft_consensus.cc:686] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 LEADER]: Becoming Leader. State: Replica: 1dbc2147296446e58d6ad07c2851d0a4, State: Running, Role: LEADER
I20260812 02:42:47.043541 20684 consensus_queue.cc:237] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:47.047998 20119 catalog_manager.cc:5495] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193). New cstate: current_term: 1 leader_uuid: "1dbc2147296446e58d6ad07c2851d0a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } health_report { overall_health: UNKNOWN } } }
I20260812 02:42:47.104007 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.098325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44416 (local address 127.17.196.193:41979)
0812 02:42:47.098818 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:47.098823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:47.098839 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:42:47.098923 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:47.098926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:47.099026 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:47.099229 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:47.099593 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.100328 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.101742 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.102191 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.102327 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.102750 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:47.103308 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:47.103467 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:47.103552 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 02:42:47.103647 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":297,"threads_started":1}
I20260812 02:42:47.128283 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.114500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:40719 (local address 127.17.196.195:40643)
0812 02:42:47.114752 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:47.114763 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:47.122415 (+ 7652us) server_negotiation.cc:407] Connection header received
0812 02:42:47.122704 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:47.122711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:47.122820 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:47.122904 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:47.123612 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.124363 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.126309 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.126885 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.127047 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.127522 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:47.127641 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:47.127756 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:47.127880 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 02:42:47.127961 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:42:47.128613 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.114396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40643 (local address 127.17.196.193:40719)
0812 02:42:47.122263 (+ 7867us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:47.122283 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:47.122502 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:47.123148 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:47.123151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:47.123171 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:42:47.123426 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:47.123433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.124525 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.124530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:47.126157 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:47.126170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.127080 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.127088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:47.127254 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.127319 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:42:47.127756 (+ 437us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:42:47.127778 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 02:42:47.127857 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 02:42:47.127998 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7666,"thread_start_us":7442,"threads_started":1}
I20260812 02:42:47.139338 20448 raft_consensus.cc:1261] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Refusing update from remote peer 1dbc2147296446e58d6ad07c2851d0a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:47.139382 20601 raft_consensus.cc:1261] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Refusing update from remote peer 1dbc2147296446e58d6ad07c2851d0a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:47.140367 20745 consensus_queue.cc:1035] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:47.141129 20684 consensus_queue.cc:1035] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:47.155678 20709 mvcc.cc:205] Tried to move back new op lower bound from 7317514514995376128 to 7317514514615726080. Current Snapshot: MvccSnapshot[applied={T|T < 7317514514995376128}]
I20260812 02:42:47.189248 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.181253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:41979 (local address 127.17.196.195:36887)
0812 02:42:47.181523 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:47.181548 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:47.181699 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:47.183640 (+ 1941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:47.183645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:47.183661 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:42:47.183829 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:47.183836 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.185189 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.185195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:47.186597 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:47.186618 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.187738 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.187748 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:47.188029 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.188115 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:42:47.188666 (+ 551us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:42:47.188678 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 02:42:47.188751 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 02:42:47.188900 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 02:42:47.191645 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:47.181636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:36887 (local address 127.17.196.193:41979)
0812 02:42:47.183073 (+ 1437us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:47.183079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:47.183099 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:42:47.183199 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:47.183222 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:47.183299 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:47.183397 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:47.183980 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.185060 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.186754 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:47.187435 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:47.187572 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:47.188353 (+ 781us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:47.188526 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:47.188614 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:47.191229 (+ 2615us) server_negotiation.cc:299] Negotiation successful
0812 02:42:47.191328 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1325}
W20260812 02:42:47.193113 20709 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 02:42:48.879655 20119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:48.880128 20119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:48.886451 20119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: Sending DeleteTablet for 3 replicas of tablet d408db78af7d4a16ab65417f0841ff50
I20260812 02:42:48.887755 20271 tablet_service.cc:1514] Processing DeleteTablet for tablet d408db78af7d4a16ab65417f0841ff50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:48 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:44408
I20260812 02:42:48.887813 20427 tablet_service.cc:1514] Processing DeleteTablet for tablet d408db78af7d4a16ab65417f0841ff50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:48 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:60110
I20260812 02:42:48.887835 20580 tablet_service.cc:1514] Processing DeleteTablet for tablet d408db78af7d4a16ab65417f0841ff50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:48 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:45468
I20260812 02:42:48.903621 20780 tablet_replica.cc:330] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: stopping tablet replica
I20260812 02:42:48.903756 20782 tablet_replica.cc:330] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: stopping tablet replica
I20260812 02:42:48.903821 20781 tablet_replica.cc:330] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: stopping tablet replica
I20260812 02:42:48.904110 20782 raft_consensus.cc:2229] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:42:48.904114 20780 raft_consensus.cc:2229] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:48.904141 20781 raft_consensus.cc:2229] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:48.904474 20780 raft_consensus.cc:2258] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:48.904502 20781 raft_consensus.cc:2258] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:48.905472 20782 raft_consensus.cc:2258] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:48.905704 20780 ts_tablet_manager.cc:1905] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:48.905788 20781 ts_tablet_manager.cc:1905] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:48.906687 20782 ts_tablet_manager.cc:1905] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:48.915217 20781 ts_tablet_manager.cc:1918] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:42:48.915220 20780 ts_tablet_manager.cc:1918] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:42:48.915413 20781 log.cc:1196] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/d408db78af7d4a16ab65417f0841ff50
I20260812 02:42:48.915521 20780 log.cc:1196] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/d408db78af7d4a16ab65417f0841ff50
I20260812 02:42:48.915720 20782 ts_tablet_manager.cc:1918] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:42:48.915892 20782 log.cc:1196] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/d408db78af7d4a16ab65417f0841ff50
I20260812 02:42:48.916316 20781 ts_tablet_manager.cc:1939] T d408db78af7d4a16ab65417f0841ff50 P fc0b4f88405f499da196fe6cfe918d5d: Deleting consensus metadata
I20260812 02:42:48.916404 20780 ts_tablet_manager.cc:1939] T d408db78af7d4a16ab65417f0841ff50 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting consensus metadata
I20260812 02:42:48.916558 20782 ts_tablet_manager.cc:1939] T d408db78af7d4a16ab65417f0841ff50 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting consensus metadata
I20260812 02:42:48.917790 20105 catalog_manager.cc:4841] TS fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093): tablet d408db78af7d4a16ab65417f0841ff50 (table test-table [id=06544b6fd3cd4b63afafb00fafeb1cbc]) successfully deleted
I20260812 02:42:48.917833 20106 catalog_manager.cc:4841] TS 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643): tablet d408db78af7d4a16ab65417f0841ff50 (table test-table [id=06544b6fd3cd4b63afafb00fafeb1cbc]) successfully deleted
I20260812 02:42:48.918314 20106 catalog_manager.cc:4841] TS 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979): tablet d408db78af7d4a16ab65417f0841ff50 (table test-table [id=06544b6fd3cd4b63afafb00fafeb1cbc]) successfully deleted
Aug 12 02:42:48 dist-test-slave-ss4n krb5kdc[20069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:42:48 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:48.966207 20790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:48.945772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39518 (local address 127.17.196.254:33733)
0812 02:42:48.946792 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:48.946801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:48.947020 (+ 219us) server_negotiation.cc:407] Connection header received
0812 02:42:48.947188 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:48.947192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:48.947260 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:48.947355 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:48.949754 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:48.950875 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:48.952758 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:48.953342 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:48.953452 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:48.959044 (+ 5592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:48.959050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:48.959053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:48.959095 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:48.962761 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:48.963692 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:48.963697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:48.963699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:48.963841 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:48.964364 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:48.964368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:48.964370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:48.964644 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:48.964887 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:48.965314 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 02:42:48.965621 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":555,"threads_started":1}
I20260812 02:42:48.971076 20119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:42:48.971856 20119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:48.979526 20271 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf444d9bc56405b9886924de6bd0241 (DEFAULT_TABLE table=test-table [id=75ec899a0fe2441082aabffd335f555e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:48.979663 20427 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf444d9bc56405b9886924de6bd0241 (DEFAULT_TABLE table=test-table [id=75ec899a0fe2441082aabffd335f555e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:48.979763 20580 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf444d9bc56405b9886924de6bd0241 (DEFAULT_TABLE table=test-table [id=75ec899a0fe2441082aabffd335f555e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:48.980254 20427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf444d9bc56405b9886924de6bd0241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:48.980276 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf444d9bc56405b9886924de6bd0241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:48.980335 20271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf444d9bc56405b9886924de6bd0241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:48.988338 20792 tablet_bootstrap.cc:492] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: Bootstrap starting.
I20260812 02:42:48.990111 20794 tablet_bootstrap.cc:492] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: Bootstrap starting.
I20260812 02:42:48.991328 20795 tablet_bootstrap.cc:492] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: Bootstrap starting.
I20260812 02:42:48.993423 20795 tablet_bootstrap.cc:654] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:48.993455 20792 tablet_bootstrap.cc:654] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:48.993454 20794 tablet_bootstrap.cc:654] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:48.999362 20792 tablet_bootstrap.cc:492] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: No bootstrap required, opened a new log
I20260812 02:42:48.999630 20792 ts_tablet_manager.cc:1397] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.005s
I20260812 02:42:49.000695 20795 tablet_bootstrap.cc:492] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: No bootstrap required, opened a new log
I20260812 02:42:49.001025 20795 ts_tablet_manager.cc:1397] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260812 02:42:49.000852 20792 raft_consensus.cc:348] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.001204 20792 raft_consensus.cc:374] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:49.001361 20794 tablet_bootstrap.cc:492] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: No bootstrap required, opened a new log
I20260812 02:42:49.001428 20792 raft_consensus.cc:729] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbc2147296446e58d6ad07c2851d0a4, State: Initialized, Role: FOLLOWER
I20260812 02:42:49.001706 20794 ts_tablet_manager.cc:1397] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.010s
I20260812 02:42:49.002003 20792 consensus_queue.cc:260] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.002653 20792 ts_tablet_manager.cc:1428] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 02:42:49.002767 20795 raft_consensus.cc:348] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.002828 20794 raft_consensus.cc:348] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.003130 20795 raft_consensus.cc:374] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:49.003177 20794 raft_consensus.cc:374] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:49.003247 20795 raft_consensus.cc:729] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0b4f88405f499da196fe6cfe918d5d, State: Initialized, Role: FOLLOWER
I20260812 02:42:49.003356 20794 raft_consensus.cc:729] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a1cf7f8ee749abb2dfc0bd861452d6, State: Initialized, Role: FOLLOWER
I20260812 02:42:49.003602 20795 consensus_queue.cc:260] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.003744 20794 consensus_queue.cc:260] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.004956 20795 ts_tablet_manager.cc:1428] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260812 02:42:49.005513 20794 ts_tablet_manager.cc:1428] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 02:42:49.016343 20800 raft_consensus.cc:482] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:49.016937 20774 raft_consensus.cc:482] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:49.016795 20800 raft_consensus.cc:504] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.017191 20774 raft_consensus.cc:504] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.018074 20800 leader_election.cc:288] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979), 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:49.018424 20774 leader_election.cc:288] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093), 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:49.018669 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf444d9bc56405b9886924de6bd0241" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbc2147296446e58d6ad07c2851d0a4" is_pre_election: true
I20260812 02:42:49.018761 20600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf444d9bc56405b9886924de6bd0241" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" is_pre_election: true
I20260812 02:42:49.019083 20600 raft_consensus.cc:2454] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 0.
I20260812 02:42:49.019084 20299 raft_consensus.cc:2454] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 0.
I20260812 02:42:49.019512 20448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf444d9bc56405b9886924de6bd0241" candidate_uuid: "1dbc2147296446e58d6ad07c2851d0a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0b4f88405f499da196fe6cfe918d5d" is_pre_election: true
I20260812 02:42:49.019557 20601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf444d9bc56405b9886924de6bd0241" candidate_uuid: "1dbc2147296446e58d6ad07c2851d0a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" is_pre_election: true
I20260812 02:42:49.019802 20375 leader_election.cc:302] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, fc0b4f88405f499da196fe6cfe918d5d; no voters:
I20260812 02:42:49.019876 20601 raft_consensus.cc:2454] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbc2147296446e58d6ad07c2851d0a4 in term 0.
I20260812 02:42:49.019899 20448 raft_consensus.cc:2454] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbc2147296446e58d6ad07c2851d0a4 in term 0.
I20260812 02:42:49.020161 20800 raft_consensus.cc:2781] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:49.020313 20800 raft_consensus.cc:482] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:49.020457 20800 raft_consensus.cc:3037] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:49.020807 20218 leader_election.cc:302] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, fc0b4f88405f499da196fe6cfe918d5d; no voters:
I20260812 02:42:49.021242 20774 raft_consensus.cc:2781] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:49.021409 20774 raft_consensus.cc:482] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:49.021623 20774 raft_consensus.cc:3037] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:49.024626 20800 raft_consensus.cc:504] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.025588 20800 leader_election.cc:288] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 election: Requested vote from peers 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979), 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:49.025444 20774 raft_consensus.cc:504] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.026109 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf444d9bc56405b9886924de6bd0241" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbc2147296446e58d6ad07c2851d0a4"
I20260812 02:42:49.026376 20601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf444d9bc56405b9886924de6bd0241" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6"
I20260812 02:42:49.026692 20601 raft_consensus.cc:3037] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:49.026774 20299 raft_consensus.cc:2379] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc0b4f88405f499da196fe6cfe918d5d in current term 1: Already voted for candidate 1dbc2147296446e58d6ad07c2851d0a4 in this term.
I20260812 02:42:49.027099 20448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf444d9bc56405b9886924de6bd0241" candidate_uuid: "1dbc2147296446e58d6ad07c2851d0a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0b4f88405f499da196fe6cfe918d5d"
I20260812 02:42:49.027364 20774 leader_election.cc:288] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [CANDIDATE]: Term 1 election: Requested vote from peers fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093), 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:49.027539 20448 raft_consensus.cc:2379] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1dbc2147296446e58d6ad07c2851d0a4 in current term 1: Already voted for candidate fc0b4f88405f499da196fe6cfe918d5d in this term.
I20260812 02:42:49.028105 20600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf444d9bc56405b9886924de6bd0241" candidate_uuid: "1dbc2147296446e58d6ad07c2851d0a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6"
I20260812 02:42:49.028776 20219 leader_election.cc:302] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4; no voters: 81a1cf7f8ee749abb2dfc0bd861452d6, fc0b4f88405f499da196fe6cfe918d5d
I20260812 02:42:49.029381 20774 raft_consensus.cc:2726] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 02:42:49.030177 20601 raft_consensus.cc:2454] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 1.
I20260812 02:42:49.030709 20375 leader_election.cc:302] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 81a1cf7f8ee749abb2dfc0bd861452d6, fc0b4f88405f499da196fe6cfe918d5d; no voters: 1dbc2147296446e58d6ad07c2851d0a4
I20260812 02:42:49.031312 20800 raft_consensus.cc:2781] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:49.032238 20800 raft_consensus.cc:686] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 1 LEADER]: Becoming Leader. State: Replica: fc0b4f88405f499da196fe6cfe918d5d, State: Running, Role: LEADER
I20260812 02:42:49.032712 20800 consensus_queue.cc:237] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.036899 20118 catalog_manager.cc:5495] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d reported cstate change: term changed from 0 to 1, leader changed from <none> to fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194). New cstate: current_term: 1 leader_uuid: "fc0b4f88405f499da196fe6cfe918d5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } health_report { overall_health: UNKNOWN } } }
I20260812 02:42:49.073604 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:49.065994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.17.196.194:35093)
0812 02:42:49.066831 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:49.066835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:49.066856 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:42:49.066917 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:49.066922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:49.066987 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:49.067079 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:49.067621 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:49.068871 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:49.070839 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:49.071637 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:49.071798 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:49.072561 (+ 763us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:49.072757 (+ 196us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:49.072842 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:49.073053 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 02:42:49.073131 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":594,"threads_started":1}
I20260812 02:42:49.086122 20601 raft_consensus.cc:1261] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Refusing update from remote peer fc0b4f88405f499da196fe6cfe918d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:49.086720 20299 raft_consensus.cc:1261] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Refusing update from remote peer fc0b4f88405f499da196fe6cfe918d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:49.087038 20801 consensus_queue.cc:1035] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 02:42:49.087903 20800 consensus_queue.cc:1035] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:49.144717 20118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:49.145004 20118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:49.149228 20118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: Sending DeleteTablet for 3 replicas of tablet 2bf444d9bc56405b9886924de6bd0241
I20260812 02:42:49.150456 20427 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf444d9bc56405b9886924de6bd0241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:49 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:60110
I20260812 02:42:49.150835 20580 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf444d9bc56405b9886924de6bd0241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:49 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:45468
I20260812 02:42:49.151078 20781 tablet_replica.cc:330] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: stopping tablet replica
I20260812 02:42:49.151160 20780 tablet_replica.cc:330] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: stopping tablet replica
I20260812 02:42:49.151115 20271 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf444d9bc56405b9886924de6bd0241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:49 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:44408
I20260812 02:42:49.151345 20780 raft_consensus.cc:2229] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:49.151525 20781 raft_consensus.cc:2229] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:42:49.151553 20782 tablet_replica.cc:330] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: stopping tablet replica
I20260812 02:42:49.151638 20780 raft_consensus.cc:2258] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:49.151805 20782 raft_consensus.cc:2229] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:49.151984 20781 raft_consensus.cc:2258] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:49.152029 20782 raft_consensus.cc:2258] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:49.153103 20781 ts_tablet_manager.cc:1905] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:49.153246 20782 ts_tablet_manager.cc:1905] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:49.153422 20118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:42:49.154171 20118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:42:49.159629 20782 ts_tablet_manager.cc:1918] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:49.159895 20782 log.cc:1196] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/2bf444d9bc56405b9886924de6bd0241
I20260812 02:42:49.160197 20780 ts_tablet_manager.cc:1905] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:49.160405 20782 ts_tablet_manager.cc:1939] T 2bf444d9bc56405b9886924de6bd0241 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting consensus metadata
I20260812 02:42:49.161635 20106 catalog_manager.cc:4841] TS 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979): tablet 2bf444d9bc56405b9886924de6bd0241 (table test-table [id=75ec899a0fe2441082aabffd335f555e]) successfully deleted
I20260812 02:42:49.162765 20271 tablet_service.cc:1467] Processing CreateTablet for tablet 8aea1243968f437cb4d898bbec9e66d6 (DEFAULT_TABLE table=test-table [id=c41c4b814ba24139afe9dc74446cf6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:49.163157 20271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aea1243968f437cb4d898bbec9e66d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:49.163560 20580 tablet_service.cc:1467] Processing CreateTablet for tablet 8aea1243968f437cb4d898bbec9e66d6 (DEFAULT_TABLE table=test-table [id=c41c4b814ba24139afe9dc74446cf6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:49.164008 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aea1243968f437cb4d898bbec9e66d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:49.168903 20792 tablet_bootstrap.cc:492] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: Bootstrap starting.
I20260812 02:42:49.171051 20792 tablet_bootstrap.cc:654] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:49.171955 20780 ts_tablet_manager.cc:1918] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:49.172152 20794 tablet_bootstrap.cc:492] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: Bootstrap starting.
I20260812 02:42:49.172328 20780 log.cc:1196] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/2bf444d9bc56405b9886924de6bd0241
I20260812 02:42:49.172694 20427 tablet_service.cc:1467] Processing CreateTablet for tablet 8aea1243968f437cb4d898bbec9e66d6 (DEFAULT_TABLE table=test-table [id=c41c4b814ba24139afe9dc74446cf6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:42:49.172875 20780 ts_tablet_manager.cc:1939] T 2bf444d9bc56405b9886924de6bd0241 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting consensus metadata
I20260812 02:42:49.174444 20106 catalog_manager.cc:4841] TS 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643): tablet 2bf444d9bc56405b9886924de6bd0241 (table test-table [id=75ec899a0fe2441082aabffd335f555e]) successfully deleted
I20260812 02:42:49.175047 20427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aea1243968f437cb4d898bbec9e66d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:49.175665 20792 tablet_bootstrap.cc:492] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: No bootstrap required, opened a new log
I20260812 02:42:49.175866 20794 tablet_bootstrap.cc:654] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:49.176004 20792 ts_tablet_manager.cc:1397] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 02:42:49.177263 20792 raft_consensus.cc:348] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.177809 20792 raft_consensus.cc:374] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:49.177970 20792 raft_consensus.cc:729] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbc2147296446e58d6ad07c2851d0a4, State: Initialized, Role: FOLLOWER
I20260812 02:42:49.178242 20792 consensus_queue.cc:260] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.179026 20792 ts_tablet_manager.cc:1428] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 02:42:49.187337 20794 tablet_bootstrap.cc:492] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: No bootstrap required, opened a new log
I20260812 02:42:49.187608 20794 ts_tablet_manager.cc:1397] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.002s
I20260812 02:42:49.188669 20794 raft_consensus.cc:348] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.189055 20794 raft_consensus.cc:374] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:49.189249 20794 raft_consensus.cc:729] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a1cf7f8ee749abb2dfc0bd861452d6, State: Initialized, Role: FOLLOWER
I20260812 02:42:49.189734 20794 consensus_queue.cc:260] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.190572 20794 ts_tablet_manager.cc:1428] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 02:42:49.192349 20781 ts_tablet_manager.cc:1918] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:42:49.192615 20781 log.cc:1196] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/2bf444d9bc56405b9886924de6bd0241
I20260812 02:42:49.193269 20781 ts_tablet_manager.cc:1939] T 2bf444d9bc56405b9886924de6bd0241 P fc0b4f88405f499da196fe6cfe918d5d: Deleting consensus metadata
I20260812 02:42:49.193627 20795 tablet_bootstrap.cc:492] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: Bootstrap starting.
I20260812 02:42:49.194630 20105 catalog_manager.cc:4841] TS fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093): tablet 2bf444d9bc56405b9886924de6bd0241 (table test-table [id=75ec899a0fe2441082aabffd335f555e]) successfully deleted
I20260812 02:42:49.195681 20795 tablet_bootstrap.cc:654] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:49.199013 20795 tablet_bootstrap.cc:492] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: No bootstrap required, opened a new log
I20260812 02:42:49.199301 20795 ts_tablet_manager.cc:1397] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 02:42:49.200098 20795 raft_consensus.cc:348] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.200368 20795 raft_consensus.cc:374] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:49.200529 20795 raft_consensus.cc:729] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0b4f88405f499da196fe6cfe918d5d, State: Initialized, Role: FOLLOWER
I20260812 02:42:49.200872 20795 consensus_queue.cc:260] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.201576 20795 ts_tablet_manager.cc:1428] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 02:42:49.326737 20801 raft_consensus.cc:482] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:42:49.327018 20801 raft_consensus.cc:504] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.327723 20801 leader_election.cc:288] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979), 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:49.328672 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aea1243968f437cb4d898bbec9e66d6" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbc2147296446e58d6ad07c2851d0a4" is_pre_election: true
I20260812 02:42:49.328714 20601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aea1243968f437cb4d898bbec9e66d6" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" is_pre_election: true
I20260812 02:42:49.328965 20299 raft_consensus.cc:2454] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 0.
I20260812 02:42:49.329250 20601 raft_consensus.cc:2454] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 0.
I20260812 02:42:49.329712 20375 leader_election.cc:302] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, fc0b4f88405f499da196fe6cfe918d5d; no voters:
I20260812 02:42:49.330159 20801 raft_consensus.cc:2781] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:42:49.330272 20801 raft_consensus.cc:482] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:42:49.330425 20801 raft_consensus.cc:3037] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:49.333237 20801 raft_consensus.cc:504] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.334113 20801 leader_election.cc:288] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 election: Requested vote from peers 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979), 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643)
I20260812 02:42:49.334622 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aea1243968f437cb4d898bbec9e66d6" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbc2147296446e58d6ad07c2851d0a4"
I20260812 02:42:49.334851 20299 raft_consensus.cc:3037] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:49.334825 20601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aea1243968f437cb4d898bbec9e66d6" candidate_uuid: "fc0b4f88405f499da196fe6cfe918d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6"
I20260812 02:42:49.335358 20601 raft_consensus.cc:3037] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:49.337972 20299 raft_consensus.cc:2454] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 1.
I20260812 02:42:49.338534 20375 leader_election.cc:302] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbc2147296446e58d6ad07c2851d0a4, fc0b4f88405f499da196fe6cfe918d5d; no voters:
I20260812 02:42:49.338840 20801 raft_consensus.cc:2781] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:49.338935 20601 raft_consensus.cc:2454] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc0b4f88405f499da196fe6cfe918d5d in term 1.
I20260812 02:42:49.339097 20801 raft_consensus.cc:686] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 1 LEADER]: Becoming Leader. State: Replica: fc0b4f88405f499da196fe6cfe918d5d, State: Running, Role: LEADER
I20260812 02:42:49.339459 20801 consensus_queue.cc:237] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } }
I20260812 02:42:49.343497 20118 catalog_manager.cc:5495] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d reported cstate change: term changed from 0 to 1, leader changed from <none> to fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194). New cstate: current_term: 1 leader_uuid: "fc0b4f88405f499da196fe6cfe918d5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc0b4f88405f499da196fe6cfe918d5d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 } health_report { overall_health: UNKNOWN } } }
I20260812 02:42:49.399394 20601 raft_consensus.cc:1261] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Refusing update from remote peer fc0b4f88405f499da196fe6cfe918d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:49.399560 20299 raft_consensus.cc:1261] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Refusing update from remote peer fc0b4f88405f499da196fe6cfe918d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:49.400126 20801 consensus_queue.cc:1035] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a1cf7f8ee749abb2dfc0bd861452d6" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:49.400576 20800 consensus_queue.cc:1035] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dbc2147296446e58d6ad07c2851d0a4" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 02:42:49.425643 20760 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 02:42:51.042323 20119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:51.042668 20119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:42:51.049751 20119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1db2815f4f494995a73f5ab29fce1b06: Sending DeleteTablet for 3 replicas of tablet 8aea1243968f437cb4d898bbec9e66d6
I20260812 02:42:51.050928 20271 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aea1243968f437cb4d898bbec9e66d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:51 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:44408
I20260812 02:42:51.051055 20427 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aea1243968f437cb4d898bbec9e66d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:51 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:60110
I20260812 02:42:51.051827 20580 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aea1243968f437cb4d898bbec9e66d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:42:51 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:45468
I20260812 02:42:51.052616 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20200
I20260812 02:42:51.067523 20846 tablet_replica.cc:330] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: stopping tablet replica
I20260812 02:42:51.067884 20846 raft_consensus.cc:2229] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:42:51.068487 20846 raft_consensus.cc:2258] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:51.069382 20845 tablet_replica.cc:330] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: stopping tablet replica
I20260812 02:42:51.069754 20847 tablet_replica.cc:330] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: stopping tablet replica
I20260812 02:42:51.069756 20846 ts_tablet_manager.cc:1905] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:51.069841 20845 raft_consensus.cc:2229] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:51.070075 20847 raft_consensus.cc:2229] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:42:51.070333 20845 raft_consensus.cc:2258] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:51.070571 20847 raft_consensus.cc:2258] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:42:51.071808 20845 ts_tablet_manager.cc:1905] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:51.074090 20847 ts_tablet_manager.cc:1905] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:42:51.077678 20846 ts_tablet_manager.cc:1918] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:42:51.077849 20846 log.cc:1196] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/8aea1243968f437cb4d898bbec9e66d6
I20260812 02:42:51.078531 20846 ts_tablet_manager.cc:1939] T 8aea1243968f437cb4d898bbec9e66d6 P fc0b4f88405f499da196fe6cfe918d5d: Deleting consensus metadata
I20260812 02:42:51.079988 20105 catalog_manager.cc:4841] TS fc0b4f88405f499da196fe6cfe918d5d (127.17.196.194:35093): tablet 8aea1243968f437cb4d898bbec9e66d6 (table test-table [id=c41c4b814ba24139afe9dc74446cf6cb]) successfully deleted
I20260812 02:42:51.081147 20845 ts_tablet_manager.cc:1918] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:42:51.081374 20845 log.cc:1196] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/8aea1243968f437cb4d898bbec9e66d6
I20260812 02:42:51.081996 20847 ts_tablet_manager.cc:1918] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:42:51.082046 20845 ts_tablet_manager.cc:1939] T 8aea1243968f437cb4d898bbec9e66d6 P 1dbc2147296446e58d6ad07c2851d0a4: Deleting consensus metadata
I20260812 02:42:51.082294 20847 log.cc:1196] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/8aea1243968f437cb4d898bbec9e66d6
I20260812 02:42:51.082988 20847 ts_tablet_manager.cc:1939] T 8aea1243968f437cb4d898bbec9e66d6 P 81a1cf7f8ee749abb2dfc0bd861452d6: Deleting consensus metadata
I20260812 02:42:51.083420 20106 catalog_manager.cc:4841] TS 1dbc2147296446e58d6ad07c2851d0a4 (127.17.196.193:41979): tablet 8aea1243968f437cb4d898bbec9e66d6 (table test-table [id=c41c4b814ba24139afe9dc74446cf6cb]) successfully deleted
I20260812 02:42:51.084164 20106 catalog_manager.cc:4841] TS 81a1cf7f8ee749abb2dfc0bd861452d6 (127.17.196.195:40643): tablet 8aea1243968f437cb4d898bbec9e66d6 (table test-table [id=c41c4b814ba24139afe9dc74446cf6cb]) successfully deleted
Aug 12 02:42:51 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:42:51.097304 20848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:51.054937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.17.196.193:41979)
0812 02:42:51.068606 (+ 13669us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:51.068615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:51.068655 (+ 40us) server_negotiation.cc:407] Connection header received
0812 02:42:51.068733 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:51.069112 (+ 379us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:51.069241 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:51.072024 (+ 2783us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:51.072301 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:51.073700 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:51.076439 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:51.077516 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:51.078827 (+ 1311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:51.089798 (+ 10971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:51.089818 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:51.089860 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:51.089940 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:51.094207 (+ 4267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:51.095134 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:51.095144 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:51.095146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:51.095335 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:51.095845 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:51.095849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:51.095851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:51.096264 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:51.096668 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:51.096800 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 02:42:51.097026 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13417,"thread_start_us":13903,"threads_started":1}
W20260812 02:42:51.098256 20342 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44424
W20260812 02:42:51.099053 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20200: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:51.099342 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 20200
I20260812 02:42:51.129129 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20353
Aug 12 02:42:51 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:42:51.150133 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:51.130125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.17.196.194:35093)
0812 02:42:51.131031 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:51.131036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:51.131055 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:42:51.131116 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:51.131121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:51.131186 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:51.131290 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:51.132967 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:51.134371 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:51.136000 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:51.136522 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:51.136642 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:51.142199 (+ 5557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:51.142221 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:51.142238 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:51.142285 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:51.146170 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:51.147061 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:51.147071 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:51.147073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:51.147219 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:51.147795 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:51.147799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:51.147802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:51.148300 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:51.148838 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:51.149290 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 02:42:51.149622 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":436,"threads_started":1}
W20260812 02:42:51.150914 20499 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60136
W20260812 02:42:51.151783 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20353: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:51.151934 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 20353
I20260812 02:42:51.183279 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20509
Aug 12 02:42:51 dist-test-slave-ss4n krb5kdc[20069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:42:51.202198 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:51.184250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.17.196.195:40643)
0812 02:42:51.184841 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:51.184847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:51.184870 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:42:51.184934 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:51.184939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:51.185009 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:51.185113 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:51.186607 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:51.187571 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:51.188891 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:51.189366 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:51.189612 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:51.194278 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:51.194296 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:51.194314 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:51.194403 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:51.198435 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:51.199410 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:51.199420 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:51.199422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:51.199702 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:51.200206 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:51.200211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:51.200213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:51.200692 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:51.200984 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:51.201416 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 02:42:51.201730 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":303,"threads_started":1}
W20260812 02:42:51.203171 20651 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45478
W20260812 02:42:51.204125 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20509: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:51.204348 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 20509
I20260812 02:42:51.235361 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20084
I20260812 02:42:51.236785 20163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:51.356766 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 20084
2026-08-12T02:42:51Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9514 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:51 dist-test-slave-ss4n krb5kdc[20855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:51 dist-test-slave-ss4n krb5kdc[20855](info): set up 2 sockets
Aug 12 02:42:51 dist-test-slave-ss4n krb5kdc[20855](info): commencing operation
krb5kdc: starting...
W20260812 02:42:52.648720 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:42:52 dist-test-slave-ss4n krb5kdc[20855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:42:52.897948 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46535
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:46535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:53.241539 20869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:53.241940 20869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:53.242131 20869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:53.250058 20869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:53.250216 20869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:53.250324 20869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:53.250433 20869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:53.250500 20869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:53.261051 20869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:46535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:53.262565 20869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:53.263921 20869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:53.276908 20878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:53.276957 20875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:53.277180 20874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:53.277781 20869 server_base.cc:1034] running on GCE node
I20260812 02:42:53.278661 20869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:53.280190 20869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:53.281430 20869 hybrid_clock.cc:648] HybridClock initialized: now 1786502573281369 us; error 40 us; skew 500 ppm
I20260812 02:42:53.282002 20869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:53 dist-test-slave-ss4n krb5kdc[20855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502573, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:53.285763 20869 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:42:53.287598 20869 webserver.cc:460] Webserver started at http://127.17.196.254:36491/ using document root <none> and password file <none>
I20260812 02:42:53.288350 20869 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:53.288511 20869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:53.288853 20869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:53.292249 20869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "1fca544f5e6e4d99bd1353ffd1be6736"
format_stamp: "Formatted at 2026-08-12 02:42:53 on dist-test-slave-ss4n"
server_key: "02fc4551b9398ad8a20de9710f32455d"
server_key_iv: "3d0edf82869472ed35e6c81e4f86d556"
server_key_version: "encryptionkey@0"
I20260812 02:42:53.293092 20869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "1fca544f5e6e4d99bd1353ffd1be6736"
format_stamp: "Formatted at 2026-08-12 02:42:53 on dist-test-slave-ss4n"
server_key: "02fc4551b9398ad8a20de9710f32455d"
server_key_iv: "3d0edf82869472ed35e6c81e4f86d556"
server_key_version: "encryptionkey@0"
I20260812 02:42:53.299826 20869 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:42:53.304421 20885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:53.306165 20869 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:42:53.306380 20869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "1fca544f5e6e4d99bd1353ffd1be6736"
format_stamp: "Formatted at 2026-08-12 02:42:53 on dist-test-slave-ss4n"
server_key: "02fc4551b9398ad8a20de9710f32455d"
server_key_iv: "3d0edf82869472ed35e6c81e4f86d556"
server_key_version: "encryptionkey@0"
I20260812 02:42:53.306646 20869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:53.330077 20869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:53.335304 20869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:53.335595 20869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:53.357506 20869 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:46535
I20260812 02:42:53.357638 20942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:46535 every 8 connection(s)
I20260812 02:42:53.359073 20869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:53.363229 20943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:53.363273 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 20869
I20260812 02:42:53.363567 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:42:53.364297 18195 external_mini_cluster.cc:1428] Setting key 28d66f7b9313a0f28827c35b25186f77
I20260812 02:42:53.377396 20943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736: Bootstrap starting.
I20260812 02:42:53.380837 20943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:53.382115 20943 log.cc:824] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:42:53 dist-test-slave-ss4n krb5kdc[20855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:53.385826 20943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736: No bootstrap required, opened a new log
I20260812 02:42:53.393858 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:53.366530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48296 (local address 127.17.196.254:46535)
0812 02:42:53.367779 (+ 1249us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:53.367801 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:53.368213 (+ 412us) server_negotiation.cc:407] Connection header received
0812 02:42:53.370783 (+ 2570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:53.370813 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:53.371361 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:53.371847 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:53.373840 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:53.375725 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:53.377302 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:53.378258 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:53.378513 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:53.383712 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:53.383723 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:53.383743 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:53.383787 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:53.388292 (+ 4505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:53.389272 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:53.389282 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:53.389291 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:53.389575 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:53.390048 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:53.390054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:53.390058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:53.390870 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:53.391210 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:53.391749 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 02:42:53.392272 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":914,"thread_start_us":620,"threads_started":1}
I20260812 02:42:53.396217 20943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca544f5e6e4d99bd1353ffd1be6736" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46535 } }
I20260812 02:42:53.396596 20943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:53.396730 20943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fca544f5e6e4d99bd1353ffd1be6736, State: Initialized, Role: FOLLOWER
I20260812 02:42:53.397387 20943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca544f5e6e4d99bd1353ffd1be6736" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46535 } }
I20260812 02:42:53.397696 20943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:53.397833 20943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:53.398017 20943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:53.400259 20943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca544f5e6e4d99bd1353ffd1be6736" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46535 } }
I20260812 02:42:53.400796 20943 leader_election.cc:302] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fca544f5e6e4d99bd1353ffd1be6736; no voters:
I20260812 02:42:53.401671 20943 leader_election.cc:288] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:53.401814 20951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:53.402573 20951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [term 1 LEADER]: Becoming Leader. State: Replica: 1fca544f5e6e4d99bd1353ffd1be6736, State: Running, Role: LEADER
I20260812 02:42:53.403121 20951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca544f5e6e4d99bd1353ffd1be6736" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46535 } }
I20260812 02:42:53.404142 20943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:53.406669 20953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fca544f5e6e4d99bd1353ffd1be6736. Latest consensus state: current_term: 1 leader_uuid: "1fca544f5e6e4d99bd1353ffd1be6736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca544f5e6e4d99bd1353ffd1be6736" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46535 } } }
I20260812 02:42:53.406831 20952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fca544f5e6e4d99bd1353ffd1be6736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca544f5e6e4d99bd1353ffd1be6736" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46535 } } }
I20260812 02:42:53.407208 20953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:53.408145 20952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:53.409324 20957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:53.416165 20957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:53.432621 20957 catalog_manager.cc:1347] Generated new cluster ID: 34d32a28b5aa42efa8e6e586c3c71847
I20260812 02:42:53.432766 20957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:53.451581 20957 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:53.453101 20957 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:53.467605 20957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fca544f5e6e4d99bd1353ffd1be6736: Generated new TSK 0
I20260812 02:42:53.468557 20957 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:42:53.538081 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46535
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:53.919344 20973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:53.919714 20973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:53.919901 20973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:53.927802 20973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:53.927979 20973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:53.928177 20973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:42:53.938795 20973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:53.940302 20973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:53.941819 20973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:53.956205 20978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:53.956419 20981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:53.956480 20979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:53.957729 20973 server_base.cc:1034] running on GCE node
I20260812 02:42:53.958371 20973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:53.959390 20973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:53.960690 20973 hybrid_clock.cc:648] HybridClock initialized: now 1786502573960643 us; error 62 us; skew 500 ppm
I20260812 02:42:53.961148 20973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:53 dist-test-slave-ss4n krb5kdc[20855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502573, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:53.965024 20973 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:42:53.966897 20973 webserver.cc:460] Webserver started at http://127.17.196.193:40621/ using document root <none> and password file <none>
I20260812 02:42:53.967725 20973 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:53.967876 20973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:53.968240 20973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:53.971743 20973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "f67ca78d24d44063ae6e259fdb7025a3"
format_stamp: "Formatted at 2026-08-12 02:42:53 on dist-test-slave-ss4n"
server_key: "d7e401e1082d3c6fab147a5aeba39ad6"
server_key_iv: "9a89b3f9f0fdf75fcb3656469ae4f825"
server_key_version: "encryptionkey@0"
I20260812 02:42:53.972680 20973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "f67ca78d24d44063ae6e259fdb7025a3"
format_stamp: "Formatted at 2026-08-12 02:42:53 on dist-test-slave-ss4n"
server_key: "d7e401e1082d3c6fab147a5aeba39ad6"
server_key_iv: "9a89b3f9f0fdf75fcb3656469ae4f825"
server_key_version: "encryptionkey@0"
I20260812 02:42:53.978991 20973 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260812 02:42:53.982936 20988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:53.984637 20973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:42:53.984838 20973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "f67ca78d24d44063ae6e259fdb7025a3"
format_stamp: "Formatted at 2026-08-12 02:42:53 on dist-test-slave-ss4n"
server_key: "d7e401e1082d3c6fab147a5aeba39ad6"
server_key_iv: "9a89b3f9f0fdf75fcb3656469ae4f825"
server_key_version: "encryptionkey@0"
I20260812 02:42:53.985069 20973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:54.010432 20973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:54.015753 20973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:54.016083 20973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:54.017179 20973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:54.018616 20973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:54.018699 20973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:54.018862 20973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:54.018927 20973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:54.056324 20973 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:36925
I20260812 02:42:54.056588 21107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:36925 every 8 connection(s)
I20260812 02:42:54.057749 20973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:42:54.066884 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 20973
I20260812 02:42:54.067167 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:42:54.067821 18195 external_mini_cluster.cc:1428] Setting key fdce2bcb22071645813e5070c189b0fc
Aug 12 02:42:54 dist-test-slave-ss4n krb5kdc[20855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502573, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:54.079177 21110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:54.060731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:33069 (local address 127.17.196.254:46535)
0812 02:42:54.061283 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:54.061288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:54.062241 (+ 953us) server_negotiation.cc:407] Connection header received
0812 02:42:54.064235 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:54.064241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:54.064334 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:54.064455 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:54.066110 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:54.066855 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:54.068593 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:54.069035 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:54.069277 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:54.073064 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:54.073069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:54.073073 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:54.073108 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:54.075843 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:54.076663 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:54.076668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:54.076670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:54.076799 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:54.077260 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:54.077269 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:54.077271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:54.077632 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:54.077813 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:54.078535 (+ 722us) server_negotiation.cc:299] Negotiation successful
0812 02:42:54.078764 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":440,"thread_start_us":351,"threads_started":1}
I20260812 02:42:54.080199 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:54.061188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46535 (local address 127.17.196.193:33069)
0812 02:42:54.062050 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:54.062097 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:54.063945 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:54.064710 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:54.064718 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:54.065238 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:54.065925 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:54.065943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:54.067020 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:54.067027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:54.068435 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:54.068448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:54.069286 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:54.069294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:54.069639 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:54.070366 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:54.070392 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:54.072813 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:54.076012 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:54.076023 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:54.076041 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:54.076517 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:54.076916 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:54.076921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:54.076925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:54.077127 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:54.077761 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:54.077769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:54.078151 (+ 382us) client_negotiation.cc:769] Sending connection context
0812 02:42:54.078427 (+ 276us) client_negotiation.cc:240] Negotiation successful
0812 02:42:54.079009 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":413,"threads_started":1}
I20260812 02:42:54.082006 21108 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46535
I20260812 02:42:54.082377 21108 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:54.083357 21108 heartbeater.cc:511] Master 127.17.196.254:46535 requested a full tablet report, sending...
I20260812 02:42:54.086043 20903 ts_manager.cc:194] Registered new tserver with Master: f67ca78d24d44063ae6e259fdb7025a3 (127.17.196.193:36925)
I20260812 02:42:54.088425 20903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:33069
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:42:54.125640 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46535
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:54.473820 21115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:54.474130 21115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:54.474228 21115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:54.481835 21115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:54.481979 21115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:54.482201 21115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:42:54.492863 21115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:54.494380 21115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:54.495975 21115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:54.510483 21120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:54.510449 21121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:54.510437 21123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:54.512356 21115 server_base.cc:1034] running on GCE node
I20260812 02:42:54.513084 21115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:54.514503 21115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:54.515782 21115 hybrid_clock.cc:648] HybridClock initialized: now 1786502574515732 us; error 49 us; skew 500 ppm
I20260812 02:42:54.516180 21115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:54 dist-test-slave-ss4n krb5kdc[20855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502574, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:54.520038 21115 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:42:54.521894 21115 webserver.cc:460] Webserver started at http://127.17.196.194:45391/ using document root <none> and password file <none>
I20260812 02:42:54.522691 21115 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:54.522852 21115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:54.523196 21115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:54.527154 21115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "ea03e06d608c4bc09a1a57c72117c660"
format_stamp: "Formatted at 2026-08-12 02:42:54 on dist-test-slave-ss4n"
server_key: "cc5fc22473890d6519d58368b15a799d"
server_key_iv: "8fa9b3b7ad52496d9ece0f19fc234f0b"
server_key_version: "encryptionkey@0"
I20260812 02:42:54.528139 21115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "ea03e06d608c4bc09a1a57c72117c660"
format_stamp: "Formatted at 2026-08-12 02:42:54 on dist-test-slave-ss4n"
server_key: "cc5fc22473890d6519d58368b15a799d"
server_key_iv: "8fa9b3b7ad52496d9ece0f19fc234f0b"
server_key_version: "encryptionkey@0"
I20260812 02:42:54.534484 21115 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260812 02:42:54.538689 21130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:54.540635 21115 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:42:54.540889 21115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "ea03e06d608c4bc09a1a57c72117c660"
format_stamp: "Formatted at 2026-08-12 02:42:54 on dist-test-slave-ss4n"
server_key: "cc5fc22473890d6519d58368b15a799d"
server_key_iv: "8fa9b3b7ad52496d9ece0f19fc234f0b"
server_key_version: "encryptionkey@0"
I20260812 02:42:54.541144 21115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:54.563620 21115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:54.568100 21115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:54.568409 21115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:54.569659 21115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:54.571126 21115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:54.571236 21115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:54.571390 21115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:54.571465 21115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:54.610066 21115 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:33715
I20260812 02:42:54.610116 21256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:33715 every 8 connection(s)
I20260812 02:42:54.611625 21115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:42:54.620219 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 21115
I20260812 02:42:54.620458 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:42:54.621019 18195 external_mini_cluster.cc:1428] Setting key e675e80e59a3274f33ffa9429b7053b7
Aug 12 02:42:54 dist-test-slave-ss4n krb5kdc[20855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502574, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:54.634352 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:54.614313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:57193 (local address 127.17.196.254:46535)
0812 02:42:54.614777 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:54.614781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:54.615986 (+ 1205us) server_negotiation.cc:407] Connection header received
0812 02:42:54.618033 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:54.618038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:54.618142 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:54.618269 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:54.619867 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:54.620762 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:54.622696 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:54.623293 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:54.623467 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:54.627180 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:54.627186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:54.627190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:54.627224 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:54.630656 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:54.631547 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:54.631556 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:54.631562 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:54.631744 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:54.632208 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:54.632216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:54.632222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:54.632626 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:54.632798 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:54.633565 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 02:42:54.633834 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260812 02:42:54.635136 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:54.615045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46535 (local address 127.17.196.194:57193)
0812 02:42:54.615813 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:54.615853 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:54.617733 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:54.618538 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:54.618547 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:54.619083 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:54.619680 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:54.619698 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:54.620948 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:54.620952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:54.622514 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:54.622538 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:54.623530 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:54.623539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:54.623805 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:54.624567 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:54.624591 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:54.626906 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:54.630847 (+ 3941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:54.630856 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:54.630872 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:54.631385 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:54.631850 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:54.631854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:54.631856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:54.632076 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:54.632758 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:54.632765 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:54.633132 (+ 367us) client_negotiation.cc:769] Sending connection context
0812 02:42:54.633378 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 02:42:54.633879 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":287,"threads_started":1}
I20260812 02:42:54.636889 21257 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46535
I20260812 02:42:54.637240 21257 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:54.638077 21257 heartbeater.cc:511] Master 127.17.196.254:46535 requested a full tablet report, sending...
I20260812 02:42:54.639674 20903 ts_manager.cc:194] Registered new tserver with Master: ea03e06d608c4bc09a1a57c72117c660 (127.17.196.194:33715)
I20260812 02:42:54.640609 20903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:57193
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:42:54.678717 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46535
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:55.047937 21263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:55.048259 21263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:55.048427 21263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:55.055945 21263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:55.056128 21263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:55.056327 21263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:42:55.066561 21263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:55.068017 21263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:55.069396 21263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:55.083209 21268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:55.083209 21271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:55.083141 21269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:55.084283 21263 server_base.cc:1034] running on GCE node
I20260812 02:42:55.084909 21263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:55.086024 21263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:55.087306 21263 hybrid_clock.cc:648] HybridClock initialized: now 1786502575087246 us; error 54 us; skew 500 ppm
I20260812 02:42:55.087644 21263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:55 dist-test-slave-ss4n krb5kdc[20855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502575, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:55.091282 21263 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:42:55.092926 21108 heartbeater.cc:503] Master 127.17.196.254:46535 was elected leader, sending a full tablet report...
I20260812 02:42:55.093176 21263 webserver.cc:460] Webserver started at http://127.17.196.195:37865/ using document root <none> and password file <none>
I20260812 02:42:55.094048 21263 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:55.094213 21263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:55.094558 21263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:55.097788 21263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "499348beaf924de89a9c2421e967883e"
format_stamp: "Formatted at 2026-08-12 02:42:55 on dist-test-slave-ss4n"
server_key: "107fe138c29f5321de4b41e8303fce44"
server_key_iv: "2a2bbb76949cf58b4ff43f36f60e03e8"
server_key_version: "encryptionkey@0"
I20260812 02:42:55.098634 21263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "499348beaf924de89a9c2421e967883e"
format_stamp: "Formatted at 2026-08-12 02:42:55 on dist-test-slave-ss4n"
server_key: "107fe138c29f5321de4b41e8303fce44"
server_key_iv: "2a2bbb76949cf58b4ff43f36f60e03e8"
server_key_version: "encryptionkey@0"
I20260812 02:42:55.104563 21263 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:42:55.109056 21280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:55.110738 21263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:42:55.110934 21263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "499348beaf924de89a9c2421e967883e"
format_stamp: "Formatted at 2026-08-12 02:42:55 on dist-test-slave-ss4n"
server_key: "107fe138c29f5321de4b41e8303fce44"
server_key_iv: "2a2bbb76949cf58b4ff43f36f60e03e8"
server_key_version: "encryptionkey@0"
I20260812 02:42:55.111181 21263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:55.124763 21263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:55.129132 21263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:55.129451 21263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:55.130559 21263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:42:55.132001 21263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:42:55.132110 21263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:55.132272 21263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:42:55.132346 21263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:55.169399 21263 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:36279
I20260812 02:42:55.169450 21406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:36279 every 8 connection(s)
I20260812 02:42:55.170827 21263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:42:55.174297 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 21263
I20260812 02:42:55.174538 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:42:55.175092 18195 external_mini_cluster.cc:1428] Setting key 3a55cb12e8b5790bf4616bc21a15e46e
Aug 12 02:42:55 dist-test-slave-ss4n krb5kdc[20855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502575, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:55.195575 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:55.174154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:46549 (local address 127.17.196.254:46535)
0812 02:42:55.174823 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:55.174830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:55.176016 (+ 1186us) server_negotiation.cc:407] Connection header received
0812 02:42:55.178371 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:55.178375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:55.178453 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:55.178552 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:55.180347 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:55.181376 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:55.183220 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:55.183680 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:55.183808 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:55.187753 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:55.187759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:55.187762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:55.187799 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:55.192054 (+ 4255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:55.192957 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:55.192962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:55.192966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:55.193107 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:55.193670 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:55.193675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:55.193677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:55.193998 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:55.194137 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:55.194978 (+ 841us) server_negotiation.cc:299] Negotiation successful
0812 02:42:55.195179 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":290,"threads_started":1}
I20260812 02:42:55.196798 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:55.174809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46535 (local address 127.17.196.195:46549)
0812 02:42:55.175805 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:55.175871 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:55.178091 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:55.178869 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:55.178878 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:55.179434 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:55.180166 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:55.180186 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:55.181598 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:55.181602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:55.183011 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:55.183024 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:55.183909 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:55.183914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:55.184140 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:55.185093 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:55.185118 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:55.187522 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:55.192281 (+ 4759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:55.192289 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:55.192304 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:55.192798 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:55.193212 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:55.193216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:55.193218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:55.193418 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:55.194134 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:55.194140 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:55.194568 (+ 428us) client_negotiation.cc:769] Sending connection context
0812 02:42:55.194822 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 02:42:55.195259 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":410,"threads_started":1}
I20260812 02:42:55.198752 21408 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46535
I20260812 02:42:55.199127 21408 heartbeater.cc:464] Registering TS with master...
I20260812 02:42:55.199884 21408 heartbeater.cc:511] Master 127.17.196.254:46535 requested a full tablet report, sending...
I20260812 02:42:55.202184 20903 ts_manager.cc:194] Registered new tserver with Master: 499348beaf924de89a9c2421e967883e (127.17.196.195:36279)
I20260812 02:42:55.203405 20903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:46549
I20260812 02:42:55.204573 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 02:42:55.249655 21410 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:55.245440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.17.196.254:46535)
0812 02:42:55.245728 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:55.245737 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:55.246379 (+ 642us) server_negotiation.cc:407] Connection header received
0812 02:42:55.246717 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:55.246727 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:55.246836 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:55.246982 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:55.249275 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48306: BlockingRecv error: recv got EOF from 127.0.0.1:48306 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 02:42:55.253703 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20973
W20260812 02:42:55.257611 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20973: Not authorized: Client connection negotiation failed: client connection to 127.17.196.193:36925: server requires authentication, but client does not have Kerberos credentials available
I20260812 02:42:55.257812 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 20973
I20260812 02:42:55.278822 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21115
W20260812 02:42:55.282243 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21115: Not authorized: Client connection negotiation failed: client connection to 127.17.196.194:33715: server requires authentication, but client does not have Kerberos credentials available
I20260812 02:42:55.282466 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 21115
I20260812 02:42:55.309666 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21263
W20260812 02:42:55.313117 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21263: Not authorized: Client connection negotiation failed: client connection to 127.17.196.195:36279: server requires authentication, but client does not have Kerberos credentials available
I20260812 02:42:55.313308 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 21263
I20260812 02:42:55.333248 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 20869
I20260812 02:42:55.334748 20938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:55.442782 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 20869
2026-08-12T02:42:55Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4080 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:55 dist-test-slave-ss4n krb5kdc[21430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:55 dist-test-slave-ss4n krb5kdc[21430](info): set up 2 sockets
Aug 12 02:42:55 dist-test-slave-ss4n krb5kdc[21430](info): commencing operation
krb5kdc: starting...
W20260812 02:42:56.698860 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:42:56 dist-test-slave-ss4n krb5kdc[21430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:42:56.953990 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39711
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:39711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:57.295717 21443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:57.296023 21443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:57.296188 21443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:57.303717 21443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:57.303843 21443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:57.303925 21443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:57.304018 21443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:57.304131 21443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:57.314600 21443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:39711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:57.316087 21443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:57.317466 21443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:57.330801 21448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:57.330754 21449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:57.330737 21451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:57.331857 21443 server_base.cc:1034] running on GCE node
I20260812 02:42:57.332767 21443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:57.334501 21443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:57.335750 21443 hybrid_clock.cc:648] HybridClock initialized: now 1786502577335682 us; error 56 us; skew 500 ppm
I20260812 02:42:57.336148 21443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:57 dist-test-slave-ss4n krb5kdc[21430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502577, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:57.345743 21443 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:42:57.347962 21443 webserver.cc:460] Webserver started at http://127.17.196.254:36627/ using document root <none> and password file <none>
I20260812 02:42:57.348812 21443 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:57.348990 21443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:57.349370 21443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:57.352995 21443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "b746267884154d36ad6ba3ae68610db8"
format_stamp: "Formatted at 2026-08-12 02:42:57 on dist-test-slave-ss4n"
server_key: "3723d356c02b4a5f9bb4cb3b95359a93"
server_key_iv: "d0530914f5797fb1a3acbc73693c48a7"
server_key_version: "encryptionkey@0"
I20260812 02:42:57.354033 21443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "b746267884154d36ad6ba3ae68610db8"
format_stamp: "Formatted at 2026-08-12 02:42:57 on dist-test-slave-ss4n"
server_key: "3723d356c02b4a5f9bb4cb3b95359a93"
server_key_iv: "d0530914f5797fb1a3acbc73693c48a7"
server_key_version: "encryptionkey@0"
I20260812 02:42:57.360437 21443 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260812 02:42:57.364691 21458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:57.366250 21443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:42:57.366474 21443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "b746267884154d36ad6ba3ae68610db8"
format_stamp: "Formatted at 2026-08-12 02:42:57 on dist-test-slave-ss4n"
server_key: "3723d356c02b4a5f9bb4cb3b95359a93"
server_key_iv: "d0530914f5797fb1a3acbc73693c48a7"
server_key_version: "encryptionkey@0"
I20260812 02:42:57.366722 21443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:57.381183 21443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:57.386929 21443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:57.387279 21443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:57.407315 21443 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:39711
I20260812 02:42:57.407336 21513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:39711 every 8 connection(s)
I20260812 02:42:57.409036 21443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:57.413192 21514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:57.418479 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 21443
I20260812 02:42:57.418716 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:42:57.419575 18195 external_mini_cluster.cc:1428] Setting key 1d09f97cea016075b19ee111bf1fb0b9
I20260812 02:42:57.425544 21514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8: Bootstrap starting.
I20260812 02:42:57.429407 21514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:57.430785 21514 log.cc:824] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:57.434628 21514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8: No bootstrap required, opened a new log
Aug 12 02:42:57 dist-test-slave-ss4n krb5kdc[21430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:57.442723 21514 raft_consensus.cc:348] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b746267884154d36ad6ba3ae68610db8" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39711 } }
I20260812 02:42:57.443171 21514 raft_consensus.cc:374] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:57.443320 21514 raft_consensus.cc:729] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b746267884154d36ad6ba3ae68610db8, State: Initialized, Role: FOLLOWER
I20260812 02:42:57.444053 21514 consensus_queue.cc:260] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b746267884154d36ad6ba3ae68610db8" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39711 } }
I20260812 02:42:57.444408 21514 raft_consensus.cc:388] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:57.444576 21514 raft_consensus.cc:482] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:57.444779 21514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:57.445753 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:57.421738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40870 (local address 127.17.196.254:39711)
0812 02:42:57.422672 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:57.422688 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:57.422733 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:42:57.424513 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:57.424539 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:57.424980 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:57.425352 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:57.427367 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:57.428760 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:57.430676 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:57.431684 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:57.431892 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:57.436807 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:57.436820 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:57.436841 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:57.436888 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:57.440566 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:57.441446 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:57.441459 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:57.441466 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:57.441730 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:57.442210 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:57.442215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:57.442218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:57.442875 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:57.443270 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:57.443796 (+ 526us) server_negotiation.cc:299] Negotiation successful
0812 02:42:57.444310 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":430,"threads_started":1}
I20260812 02:42:57.447395 21514 raft_consensus.cc:504] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b746267884154d36ad6ba3ae68610db8" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39711 } }
I20260812 02:42:57.447932 21514 leader_election.cc:302] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b746267884154d36ad6ba3ae68610db8; no voters:
I20260812 02:42:57.448632 21514 leader_election.cc:288] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:57.448758 21519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:57.450327 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:57.450460 21519 raft_consensus.cc:686] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [term 1 LEADER]: Becoming Leader. State: Replica: b746267884154d36ad6ba3ae68610db8, State: Running, Role: LEADER
I20260812 02:42:57.451007 21519 consensus_queue.cc:237] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b746267884154d36ad6ba3ae68610db8" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39711 } }
I20260812 02:42:57.451237 21514 sys_catalog.cc:564] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:57.454041 21522 sys_catalog.cc:455] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b746267884154d36ad6ba3ae68610db8. Latest consensus state: current_term: 1 leader_uuid: "b746267884154d36ad6ba3ae68610db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b746267884154d36ad6ba3ae68610db8" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39711 } } }
I20260812 02:42:57.454517 21522 sys_catalog.cc:458] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:57.455267 21521 sys_catalog.cc:455] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b746267884154d36ad6ba3ae68610db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b746267884154d36ad6ba3ae68610db8" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39711 } } }
I20260812 02:42:57.455642 21521 sys_catalog.cc:458] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:57.460779 21526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:57.469739 21526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:57.481508 21526 catalog_manager.cc:1347] Generated new cluster ID: efb337b99014408e8e8c00fdbff6d650
I20260812 02:42:57.481642 21526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:57.494659 21526 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:57.496877 21526 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:57.508368 21526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b746267884154d36ad6ba3ae68610db8: Generated new TSK 0
I20260812 02:42:57.509802 21526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:57.524287 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:57.516483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.17.196.254:39711)
0812 02:42:57.516662 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:57.516669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:57.517168 (+ 499us) server_negotiation.cc:407] Connection header received
0812 02:42:57.517394 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:57.517398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:57.517470 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:57.517619 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:57.518706 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:57.519477 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:57.520782 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:57.521225 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:57.521376 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:57.522645 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:57.522650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:57.522654 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:57.523346 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:57.523445 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:57.523491 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:57.523626 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:57.523828 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 02:42:57.523932 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 02:42:57.533617 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21443
I20260812 02:42:57.535250 21509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:57.626678 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 21443
2026-08-12T02:42:57Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2178 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:57 dist-test-slave-ss4n krb5kdc[21553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:57 dist-test-slave-ss4n krb5kdc[21553](info): set up 2 sockets
Aug 12 02:42:57 dist-test-slave-ss4n krb5kdc[21553](info): commencing operation
krb5kdc: starting...
W20260812 02:42:58.936766 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:42:59 dist-test-slave-ss4n krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:42:59.186743 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39997
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:39997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:59.623755 21567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:59.624141 21567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:59.624318 21567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:59.631690 21567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:59.631871 21567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:59.631991 21567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:59.632084 21567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:59.632176 21567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:59.642335 21567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:39997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:59.643810 21567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:59.645207 21567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:59.657910 21575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:59.657950 21573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:59.657977 21572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:59.659401 21567 server_base.cc:1034] running on GCE node
I20260812 02:42:59.660319 21567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:59.661903 21567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:59.663172 21567 hybrid_clock.cc:648] HybridClock initialized: now 1786502579663114 us; error 46 us; skew 500 ppm
I20260812 02:42:59.663650 21567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:59 dist-test-slave-ss4n krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502579, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:59.667171 21567 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:42:59.669013 21567 webserver.cc:460] Webserver started at http://127.17.196.254:44835/ using document root <none> and password file <none>
I20260812 02:42:59.669847 21567 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:59.670001 21567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:59.670337 21567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:59.673640 21567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846"
format_stamp: "Formatted at 2026-08-12 02:42:59 on dist-test-slave-ss4n"
server_key: "2466441fb6d83295754900e208adb045"
server_key_iv: "60ea3775e03a16aa8b4fdf33cac9e8fe"
server_key_version: "encryptionkey@0"
I20260812 02:42:59.674477 21567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846"
format_stamp: "Formatted at 2026-08-12 02:42:59 on dist-test-slave-ss4n"
server_key: "2466441fb6d83295754900e208adb045"
server_key_iv: "60ea3775e03a16aa8b4fdf33cac9e8fe"
server_key_version: "encryptionkey@0"
I20260812 02:42:59.680311 21567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:42:59.684211 21585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:59.685945 21567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:42:59.686154 21567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846"
format_stamp: "Formatted at 2026-08-12 02:42:59 on dist-test-slave-ss4n"
server_key: "2466441fb6d83295754900e208adb045"
server_key_iv: "60ea3775e03a16aa8b4fdf33cac9e8fe"
server_key_version: "encryptionkey@0"
I20260812 02:42:59.686372 21567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:59.714761 21567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:59.719923 21567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:59.720275 21567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:59.741631 21567 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:39997
I20260812 02:42:59.741729 21638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:39997 every 8 connection(s)
I20260812 02:42:59.743211 21567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:59.747301 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 21567
I20260812 02:42:59.747359 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:59.747546 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:42:59.748237 18195 external_mini_cluster.cc:1428] Setting key 0e4c6e359cf218bf5f632ac822879a6f
I20260812 02:42:59.762864 21640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846: Bootstrap starting.
Aug 12 02:42:59 dist-test-slave-ss4n krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:42:59.767311 21640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:59.768887 21640 log.cc:824] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:59.773128 21640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846: No bootstrap required, opened a new log
I20260812 02:42:59.778095 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:59.750560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.17.196.254:39997)
0812 02:42:59.751387 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:59.751404 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:59.751468 (+ 64us) server_negotiation.cc:407] Connection header received
0812 02:42:59.754452 (+ 2984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:59.754492 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:59.755150 (+ 658us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:59.755681 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:59.757150 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:59.759262 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:59.760697 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:59.761909 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:59.762112 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:59.767322 (+ 5210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:59.767338 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:59.767363 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:59.767421 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:59.771880 (+ 4459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:59.773009 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:59.773020 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:59.773031 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:59.773318 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:59.773994 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:59.774001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:59.774005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:59.774859 (+ 854us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:59.775249 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:59.775756 (+ 507us) server_negotiation.cc:299] Negotiation successful
0812 02:42:59.776366 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":343,"threads_started":1}
I20260812 02:42:59.784461 21640 raft_consensus.cc:348] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39997 } }
I20260812 02:42:59.784904 21640 raft_consensus.cc:374] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:59.784996 21640 raft_consensus.cc:729] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca43ed9f2a44a55bfcd1e45bfb6f846, State: Initialized, Role: FOLLOWER
I20260812 02:42:59.785694 21640 consensus_queue.cc:260] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39997 } }
I20260812 02:42:59.785939 21640 raft_consensus.cc:388] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:59.786068 21640 raft_consensus.cc:482] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:59.786211 21640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:59.789059 21640 raft_consensus.cc:504] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39997 } }
I20260812 02:42:59.789698 21640 leader_election.cc:302] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aca43ed9f2a44a55bfcd1e45bfb6f846; no voters:
I20260812 02:42:59.790346 21640 leader_election.cc:288] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:59.790668 21646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:59.791405 21646 raft_consensus.cc:686] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [term 1 LEADER]: Becoming Leader. State: Replica: aca43ed9f2a44a55bfcd1e45bfb6f846, State: Running, Role: LEADER
I20260812 02:42:59.792063 21646 consensus_queue.cc:237] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39997 } }
I20260812 02:42:59.792835 21640 sys_catalog.cc:564] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:59.794878 21648 sys_catalog.cc:455] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aca43ed9f2a44a55bfcd1e45bfb6f846. Latest consensus state: current_term: 1 leader_uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39997 } } }
I20260812 02:42:59.795262 21648 sys_catalog.cc:458] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:59.796620 21647 sys_catalog.cc:455] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca43ed9f2a44a55bfcd1e45bfb6f846" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39997 } } }
I20260812 02:42:59.797283 21647 sys_catalog.cc:458] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:59.798429 21651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:59.804039 21651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:59.813088 21651 catalog_manager.cc:1347] Generated new cluster ID: a2809c9273534598a3218fec9108c550
I20260812 02:42:59.813254 21651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:59.833448 21651 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:59.834906 21651 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:59.841526 21651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aca43ed9f2a44a55bfcd1e45bfb6f846: Generated new TSK 0
I20260812 02:42:59.842417 21651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:42:59.910787 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39997
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:00.272955 21667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:00.273286 21667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:00.273382 21667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:00.281291 21667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:00.281415 21667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:00.281630 21667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:00.293464 21667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:00.295013 21667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:00.296422 21667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:00.311306 21676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:00.311311 21673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:00.311234 21674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:00.312379 21667 server_base.cc:1034] running on GCE node
I20260812 02:43:00.313086 21667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:00.314203 21667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:00.315442 21667 hybrid_clock.cc:648] HybridClock initialized: now 1786502580315384 us; error 56 us; skew 500 ppm
I20260812 02:43:00.315780 21667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:00 dist-test-slave-ss4n krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502580, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:00.319420 21667 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:43:00.321159 21667 webserver.cc:460] Webserver started at http://127.17.196.193:44737/ using document root <none> and password file <none>
I20260812 02:43:00.321957 21667 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:00.322073 21667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:00.322353 21667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:00.325467 21667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "b99b0cb289f146619d9f57119be59e6e"
format_stamp: "Formatted at 2026-08-12 02:43:00 on dist-test-slave-ss4n"
server_key: "2ea88aee5e2183834a4b6772231791f9"
server_key_iv: "c0b0b252543b3482676fc9b1d77f5103"
server_key_version: "encryptionkey@0"
I20260812 02:43:00.326377 21667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "b99b0cb289f146619d9f57119be59e6e"
format_stamp: "Formatted at 2026-08-12 02:43:00 on dist-test-slave-ss4n"
server_key: "2ea88aee5e2183834a4b6772231791f9"
server_key_iv: "c0b0b252543b3482676fc9b1d77f5103"
server_key_version: "encryptionkey@0"
I20260812 02:43:00.331944 21667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:43:00.335724 21684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:00.337199 21667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:43:00.337399 21667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "b99b0cb289f146619d9f57119be59e6e"
format_stamp: "Formatted at 2026-08-12 02:43:00 on dist-test-slave-ss4n"
server_key: "2ea88aee5e2183834a4b6772231791f9"
server_key_iv: "c0b0b252543b3482676fc9b1d77f5103"
server_key_version: "encryptionkey@0"
I20260812 02:43:00.337680 21667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:00.364709 21667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:00.369375 21667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:00.369788 21667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:00.370854 21667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:00.372275 21667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:00.372352 21667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:00.372507 21667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:00.372584 21667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:00.409314 21667 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:45763
I20260812 02:43:00.409454 21807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:45763 every 8 connection(s)
I20260812 02:43:00.410733 21667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:00.420120 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 21667
I20260812 02:43:00.420346 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:43:00.421093 18195 external_mini_cluster.cc:1428] Setting key 0482a0c4740ba9a960614d58093dbbd3
Aug 12 02:43:00 dist-test-slave-ss4n krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502580, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:00.433959 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:00.413447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:59779 (local address 127.17.196.254:39997)
0812 02:43:00.413895 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:00.413900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:00.414967 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 02:43:00.416856 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:00.416863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:00.416961 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:00.417085 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:00.418791 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:00.419523 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:00.421295 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:00.421906 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:00.422119 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:00.426041 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:00.426050 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:00.426058 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:00.426107 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:00.429964 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:00.430874 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:00.430883 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:00.430889 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:00.431071 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:00.431841 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:00.431846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:00.431848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:00.432162 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:00.432299 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:00.433322 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0812 02:43:00.433688 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":203,"threads_started":1}
I20260812 02:43:00.435436 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:00.414054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39997 (local address 127.17.196.193:59779)
0812 02:43:00.414793 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:00.414833 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:00.416568 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:00.417346 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:00.417354 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:00.418003 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:00.418605 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:00.418623 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:00.419694 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:00.419700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:00.421075 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:00.421088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:00.422223 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:00.422233 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:00.422492 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:00.423279 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:00.423303 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:00.425733 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:00.430183 (+ 4450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:00.430192 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:00.430206 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:00.430716 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:00.431219 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:00.431225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:00.431230 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:00.431515 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:00.432317 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:00.432331 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:00.432841 (+ 510us) client_negotiation.cc:769] Sending connection context
0812 02:43:00.433190 (+ 349us) client_negotiation.cc:240] Negotiation successful
0812 02:43:00.433852 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":276,"threads_started":1}
I20260812 02:43:00.437238 21808 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39997
I20260812 02:43:00.437670 21808 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:00.438439 21808 heartbeater.cc:511] Master 127.17.196.254:39997 requested a full tablet report, sending...
I20260812 02:43:00.440697 21603 ts_manager.cc:194] Registered new tserver with Master: b99b0cb289f146619d9f57119be59e6e (127.17.196.193:45763)
I20260812 02:43:00.442451 21603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:59779
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:43:00.482033 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39997
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:00.837170 21814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:00.837550 21814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:00.837723 21814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:00.845366 21814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:00.845571 21814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:00.845769 21814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:00.855721 21814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:00.857326 21814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:00.858937 21814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:00.873445 21819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:00.873579 21820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:00.873370 21822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:00.874236 21814 server_base.cc:1034] running on GCE node
I20260812 02:43:00.874958 21814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:00.876042 21814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:00.877339 21814 hybrid_clock.cc:648] HybridClock initialized: now 1786502580877273 us; error 59 us; skew 500 ppm
I20260812 02:43:00.877756 21814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:00 dist-test-slave-ss4n krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502580, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:00.881302 21814 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:43:00.883065 21814 webserver.cc:460] Webserver started at http://127.17.196.194:46259/ using document root <none> and password file <none>
I20260812 02:43:00.883880 21814 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:00.884066 21814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:00.884428 21814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:00.887835 21814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "233f81ae44404c6a82d4e723f69e5d42"
format_stamp: "Formatted at 2026-08-12 02:43:00 on dist-test-slave-ss4n"
server_key: "a9c2015c6772990057c3a59529ac63e9"
server_key_iv: "abdc3639f2e6df110e636298b8dd46e7"
server_key_version: "encryptionkey@0"
I20260812 02:43:00.888778 21814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "233f81ae44404c6a82d4e723f69e5d42"
format_stamp: "Formatted at 2026-08-12 02:43:00 on dist-test-slave-ss4n"
server_key: "a9c2015c6772990057c3a59529ac63e9"
server_key_iv: "abdc3639f2e6df110e636298b8dd46e7"
server_key_version: "encryptionkey@0"
I20260812 02:43:00.895236 21814 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 02:43:00.899463 21832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:00.901242 21814 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260812 02:43:00.901507 21814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "233f81ae44404c6a82d4e723f69e5d42"
format_stamp: "Formatted at 2026-08-12 02:43:00 on dist-test-slave-ss4n"
server_key: "a9c2015c6772990057c3a59529ac63e9"
server_key_iv: "abdc3639f2e6df110e636298b8dd46e7"
server_key_version: "encryptionkey@0"
I20260812 02:43:00.901751 21814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:00.925021 21814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:00.930047 21814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:00.930397 21814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:00.931528 21814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:00.933032 21814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:00.933140 21814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:00.933301 21814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:00.933377 21814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:00.969213 21814 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:41039
I20260812 02:43:00.969333 21952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:41039 every 8 connection(s)
I20260812 02:43:00.970706 21814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:00.978592 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 21814
I20260812 02:43:00.978803 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:43:00.979414 18195 external_mini_cluster.cc:1428] Setting key 83e82b764d58b32a7de98fbf038649c3
Aug 12 02:43:00 dist-test-slave-ss4n krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502580, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:00.995744 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:00.973597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:44675 (local address 127.17.196.254:39997)
0812 02:43:00.974075 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:00.974080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:00.975077 (+ 997us) server_negotiation.cc:407] Connection header received
0812 02:43:00.977181 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:00.977188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:00.977293 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:00.977436 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:00.979885 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:00.980739 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:00.983209 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:00.983649 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:00.983765 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:00.988367 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:00.988372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:00.988376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:00.988411 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:00.991814 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:00.992702 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:00.992710 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:00.992720 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:00.992876 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:00.993549 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:00.993553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:00.993555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:00.993876 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:00.994071 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:00.994968 (+ 897us) server_negotiation.cc:299] Negotiation successful
0812 02:43:00.995243 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":262,"threads_started":1}
I20260812 02:43:00.997128 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:00.974053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39997 (local address 127.17.196.194:44675)
0812 02:43:00.974867 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:00.974917 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:00.976848 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:00.977769 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:00.977779 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:00.978710 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:00.979648 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:00.979680 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:00.980999 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:00.981003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:00.983024 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:00.983043 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:00.983918 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:00.983929 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:00.984293 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:00.985093 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:00.985144 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:00.988077 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:00.992012 (+ 3935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:00.992025 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:00.992045 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:00.992547 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:00.993022 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:00.993029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:00.993033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:00.993352 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:00.994057 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:00.994068 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:00.994516 (+ 448us) client_negotiation.cc:769] Sending connection context
0812 02:43:00.994859 (+ 343us) client_negotiation.cc:240] Negotiation successful
0812 02:43:00.995790 (+ 931us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":351,"threads_started":1}
I20260812 02:43:00.998863 21953 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39997
I20260812 02:43:00.999228 21953 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:01.000262 21953 heartbeater.cc:511] Master 127.17.196.254:39997 requested a full tablet report, sending...
I20260812 02:43:01.002205 21603 ts_manager.cc:194] Registered new tserver with Master: 233f81ae44404c6a82d4e723f69e5d42 (127.17.196.194:41039)
I20260812 02:43:01.003208 21603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:44675
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:43:01.038053 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39997
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:01.389034 21959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:01.389365 21959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:01.389475 21959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:01.397472 21959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:01.397629 21959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:01.397744 21959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:01.408258 21959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:01.409760 21959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:01.411183 21959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:01.425333 21965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:01.425403 21966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:01.426087 21959 server_base.cc:1034] running on GCE node
W20260812 02:43:01.426280 21968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:01.427007 21959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:01.428043 21959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:01.429298 21959 hybrid_clock.cc:648] HybridClock initialized: now 1786502581429242 us; error 43 us; skew 500 ppm
I20260812 02:43:01.429728 21959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502581, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:01.433172 21959 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:43:01.434834 21959 webserver.cc:460] Webserver started at http://127.17.196.195:32785/ using document root <none> and password file <none>
I20260812 02:43:01.435577 21959 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:01.435735 21959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:01.436070 21959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:01.439260 21959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "63e7a97853194356bb2b1168f7de040d"
format_stamp: "Formatted at 2026-08-12 02:43:01 on dist-test-slave-ss4n"
server_key: "9edd8a210dcdd19750459cfb60ed06c7"
server_key_iv: "db4d416179762c8a4354571afc3ca5b0"
server_key_version: "encryptionkey@0"
I20260812 02:43:01.440099 21959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "63e7a97853194356bb2b1168f7de040d"
format_stamp: "Formatted at 2026-08-12 02:43:01 on dist-test-slave-ss4n"
server_key: "9edd8a210dcdd19750459cfb60ed06c7"
server_key_iv: "db4d416179762c8a4354571afc3ca5b0"
server_key_version: "encryptionkey@0"
I20260812 02:43:01.445652 21808 heartbeater.cc:503] Master 127.17.196.254:39997 was elected leader, sending a full tablet report...
I20260812 02:43:01.445861 21959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:43:01.450014 21975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:01.451695 21959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:43:01.451896 21959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "63e7a97853194356bb2b1168f7de040d"
format_stamp: "Formatted at 2026-08-12 02:43:01 on dist-test-slave-ss4n"
server_key: "9edd8a210dcdd19750459cfb60ed06c7"
server_key_iv: "db4d416179762c8a4354571afc3ca5b0"
server_key_version: "encryptionkey@0"
I20260812 02:43:01.452116 21959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:01.464701 21959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:01.469281 21959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:01.469668 21959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:01.470685 21959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:01.472056 21959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:01.472172 21959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:01.472323 21959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:01.472419 21959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:01.508953 21959 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:46249
I20260812 02:43:01.508999 22098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:46249 every 8 connection(s)
I20260812 02:43:01.510428 21959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:01.512991 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 21959
I20260812 02:43:01.513206 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:43:01.513876 18195 external_mini_cluster.cc:1428] Setting key b4f7a00b27e7fbbd7a6fb6d14ac72ced
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502581, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:01.532805 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:01.513284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:48633 (local address 127.17.196.254:39997)
0812 02:43:01.513829 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:01.513836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:01.515055 (+ 1219us) server_negotiation.cc:407] Connection header received
0812 02:43:01.517243 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:01.517250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:01.517358 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:01.517451 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:01.519041 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.519825 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.521892 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.522311 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.522412 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:01.526174 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:01.526180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:01.526184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:01.526224 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:01.529833 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.530577 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.530581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.530583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.530719 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.531116 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.531119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.531122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.531405 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:01.531606 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:01.532313 (+ 707us) server_negotiation.cc:299] Negotiation successful
0812 02:43:01.532526 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":256,"threads_started":1}
I20260812 02:43:01.533828 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:01.513981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39997 (local address 127.17.196.195:48633)
0812 02:43:01.514871 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:01.514932 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:01.516982 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:01.517753 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:01.517762 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:01.518293 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:01.518872 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:01.518889 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.520033 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.520039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:01.521740 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:01.521757 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.522530 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.522536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:01.522746 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:01.523473 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:01.523497 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:01.525908 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:01.529970 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:01.529978 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:01.529991 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:01.530431 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:01.530823 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:01.530826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:01.530828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:01.531008 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:01.531600 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:01.531608 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:01.531932 (+ 324us) client_negotiation.cc:769] Sending connection context
0812 02:43:01.532189 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 02:43:01.532584 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":356,"threads_started":1}
I20260812 02:43:01.535315 22099 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39997
I20260812 02:43:01.535662 22099 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:01.536408 22099 heartbeater.cc:511] Master 127.17.196.254:39997 requested a full tablet report, sending...
I20260812 02:43:01.537878 21603 ts_manager.cc:194] Registered new tserver with Master: 63e7a97853194356bb2b1168f7de040d (127.17.196.195:46249)
I20260812 02:43:01.538702 21603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:48633
I20260812 02:43:01.542239 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[21553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502581, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502581, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:01.600512 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:01.584542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34520 (local address 127.17.196.254:39997)
0812 02:43:01.584741 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:01.584746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:01.585281 (+ 535us) server_negotiation.cc:407] Connection header received
0812 02:43:01.585426 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:01.585430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:01.585618 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:01.585720 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:01.587110 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.587870 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.589137 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.589754 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.589879 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:01.594397 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:01.594403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:01.594407 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:01.594449 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:01.597312 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.598174 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.598178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.598180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.598370 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.598875 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.598879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.598881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.599261 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:01.599513 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:01.599930 (+ 417us) server_negotiation.cc:299] Negotiation successful
0812 02:43:01.600141 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 02:43:01.601207 21603 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34520
I20260812 02:43:01.603068 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21667
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502581, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:43:01.619611 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:01.604096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33082 (local address 127.17.196.193:45763)
0812 02:43:01.604727 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:01.604735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:01.604760 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:43:01.604823 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:01.604830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:01.605046 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:01.605248 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:01.606556 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.607595 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.608810 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.609399 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.609577 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:01.613692 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:01.613705 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:01.613721 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:01.613759 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:01.616593 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.617295 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.617303 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.617305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.617438 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.617925 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.617930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.617934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.618339 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:01.618661 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:01.618960 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 02:43:01.619242 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":288,"threads_started":1}
W20260812 02:43:01.620918 21803 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33082
W20260812 02:43:01.622218 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21667: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:43:01.622483 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 21667
I20260812 02:43:01.643582 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21814
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502581, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:43:01.660759 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:01.644599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.17.196.194:41039)
0812 02:43:01.645195 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:01.645204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:01.645231 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:43:01.645294 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:01.645300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:01.645755 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:01.645898 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:01.647178 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.648179 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.649427 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.650154 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.650255 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:01.654567 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:01.654580 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:01.654596 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:01.654658 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:01.657833 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.658585 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.658593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.658595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.658758 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.659133 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.659136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.659138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.659502 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:01.659752 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:01.660101 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 02:43:01.660311 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":315,"threads_started":1}
W20260812 02:43:01.662192 21948 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38966
W20260812 02:43:01.663498 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21814: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:43:01.663653 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 21814
I20260812 02:43:01.683923 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21959
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[21553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502581, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:43:01.702157 22114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:01.685107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.17.196.195:46249)
0812 02:43:01.685876 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:01.685883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:01.685914 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:43:01.685976 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:01.685983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:01.686197 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:01.686325 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:01.687671 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.688733 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.690179 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:01.690921 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:01.691080 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:01.696005 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:01.696019 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:01.696036 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:01.696075 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:01.699054 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.699799 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.699806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.699809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.699939 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:01.700354 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:01.700359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:01.700362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:01.700753 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:01.701101 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:01.701434 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 02:43:01.701766 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":419,"threads_started":1}
W20260812 02:43:01.703656 22094 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54952
W20260812 02:43:01.704918 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21959: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:43:01.705093 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 21959
I20260812 02:43:01.725441 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 21567
I20260812 02:43:01.726948 21634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:01.832733 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 21567
2026-08-12T02:43:01Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4211 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[22118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[22118](info): set up 2 sockets
Aug 12 02:43:01 dist-test-slave-ss4n krb5kdc[22118](info): commencing operation
krb5kdc: starting...
W20260812 02:43:03.030195 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.012s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:43:03 dist-test-slave-ss4n krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:43:03.282581 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46457
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:46457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:03.632630 22133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:03.633055 22133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:03.633234 22133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:03.640748 22133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:03.640942 22133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:03.641067 22133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:03.641184 22133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:03.641276 22133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:03.652315 22133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:46457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:03.654006 22133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:03.655390 22133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:03.668730 22141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:03.668655 22138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:03.668707 22139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:03.670099 22133 server_base.cc:1034] running on GCE node
I20260812 02:43:03.671144 22133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:03.672817 22133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:03.674103 22133 hybrid_clock.cc:648] HybridClock initialized: now 1786502583674061 us; error 72 us; skew 500 ppm
I20260812 02:43:03.674592 22133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:03 dist-test-slave-ss4n krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502583, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:03.682336 22133 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:43:03.684068 22133 webserver.cc:460] Webserver started at http://127.17.196.254:41647/ using document root <none> and password file <none>
I20260812 02:43:03.684852 22133 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:03.685007 22133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:03.685534 22133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:03.688817 22133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "d29678e450884bfd8dbf6f5570da860e"
format_stamp: "Formatted at 2026-08-12 02:43:03 on dist-test-slave-ss4n"
server_key: "26fc210e2cb3c28af7169237d4609249"
server_key_iv: "de8a062ce7ca634051247633c3f3e81b"
server_key_version: "encryptionkey@0"
I20260812 02:43:03.689751 22133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "d29678e450884bfd8dbf6f5570da860e"
format_stamp: "Formatted at 2026-08-12 02:43:03 on dist-test-slave-ss4n"
server_key: "26fc210e2cb3c28af7169237d4609249"
server_key_iv: "de8a062ce7ca634051247633c3f3e81b"
server_key_version: "encryptionkey@0"
I20260812 02:43:03.695425 22133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:43:03.699443 22150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:03.701161 22133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:43:03.701424 22133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "d29678e450884bfd8dbf6f5570da860e"
format_stamp: "Formatted at 2026-08-12 02:43:03 on dist-test-slave-ss4n"
server_key: "26fc210e2cb3c28af7169237d4609249"
server_key_iv: "de8a062ce7ca634051247633c3f3e81b"
server_key_version: "encryptionkey@0"
I20260812 02:43:03.701756 22133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:03.722020 22133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:03.727114 22133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:03.727479 22133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:03.749310 22133 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:46457
I20260812 02:43:03.749368 22209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:46457 every 8 connection(s)
I20260812 02:43:03.751116 22133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:03.755532 22210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:03.759873 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 22133
I20260812 02:43:03.760154 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:03.760820 18195 external_mini_cluster.cc:1428] Setting key 0cd60b240699e8a0dd3cb81dfe4ab863
I20260812 02:43:03.768610 22210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e: Bootstrap starting.
I20260812 02:43:03.772904 22210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:03.774406 22210 log.cc:824] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:43:03 dist-test-slave-ss4n krb5kdc[22118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:03.778589 22210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e: No bootstrap required, opened a new log
I20260812 02:43:03.789935 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:03.763149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45744 (local address 127.17.196.254:46457)
0812 02:43:03.764002 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:03.764016 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:03.764059 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:43:03.766729 (+ 2670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:03.766759 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:03.767199 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:03.767576 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:03.769781 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:03.771251 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:03.773375 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:03.774250 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:03.774404 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:03.779554 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:03.779568 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:03.779590 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:03.779642 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:03.784467 (+ 4825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:03.785230 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:03.785241 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:03.785254 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:03.785674 (+ 420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:03.786053 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:03.786064 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:03.786068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:03.786948 (+ 880us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:03.787304 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:03.787713 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 02:43:03.788254 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":404,"threads_started":1}
I20260812 02:43:03.791282 22210 raft_consensus.cc:348] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29678e450884bfd8dbf6f5570da860e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46457 } }
I20260812 02:43:03.791697 22210 raft_consensus.cc:374] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:03.791841 22210 raft_consensus.cc:729] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d29678e450884bfd8dbf6f5570da860e, State: Initialized, Role: FOLLOWER
I20260812 02:43:03.792479 22210 consensus_queue.cc:260] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29678e450884bfd8dbf6f5570da860e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46457 } }
I20260812 02:43:03.792758 22210 raft_consensus.cc:388] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:03.792922 22210 raft_consensus.cc:482] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:03.793138 22210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:03.795672 22210 raft_consensus.cc:504] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29678e450884bfd8dbf6f5570da860e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46457 } }
I20260812 02:43:03.796233 22210 leader_election.cc:302] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d29678e450884bfd8dbf6f5570da860e; no voters:
I20260812 02:43:03.796866 22210 leader_election.cc:288] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:03.797025 22215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:03.797840 22215 raft_consensus.cc:686] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [term 1 LEADER]: Becoming Leader. State: Replica: d29678e450884bfd8dbf6f5570da860e, State: Running, Role: LEADER
I20260812 02:43:03.798424 22215 consensus_queue.cc:237] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29678e450884bfd8dbf6f5570da860e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46457 } }
I20260812 02:43:03.799420 22210 sys_catalog.cc:564] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:03.801614 22217 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d29678e450884bfd8dbf6f5570da860e. Latest consensus state: current_term: 1 leader_uuid: "d29678e450884bfd8dbf6f5570da860e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29678e450884bfd8dbf6f5570da860e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46457 } } }
I20260812 02:43:03.801995 22217 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:03.801760 22216 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d29678e450884bfd8dbf6f5570da860e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29678e450884bfd8dbf6f5570da860e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46457 } } }
I20260812 02:43:03.802479 22216 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:03.803879 22220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:03.809700 22220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:03.820633 22220 catalog_manager.cc:1347] Generated new cluster ID: d7900835e0554f3cabb5ac0d722f3fb8
I20260812 02:43:03.820793 22220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:03.837015 22220 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:03.839006 22220 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:03.850419 22220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d29678e450884bfd8dbf6f5570da860e: Generated new TSK 0
I20260812 02:43:03.851550 22220 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:43:03.919237 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46457
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:04.296460 22237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:04.296842 22237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:04.297005 22237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:04.306504 22237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:04.306664 22237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:04.306849 22237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:04.317525 22237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:04.319036 22237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:04.320348 22237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:04.335597 22246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:04.335711 22243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:04.335714 22242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:04.336835 22237 server_base.cc:1034] running on GCE node
I20260812 02:43:04.337653 22237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:04.338824 22237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:04.340080 22237 hybrid_clock.cc:648] HybridClock initialized: now 1786502584340037 us; error 51 us; skew 500 ppm
I20260812 02:43:04.340432 22237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:04 dist-test-slave-ss4n krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502584, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:04.344115 22237 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:43:04.345853 22237 webserver.cc:460] Webserver started at http://127.17.196.193:46603/ using document root <none> and password file <none>
I20260812 02:43:04.346599 22237 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:04.346750 22237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:04.347085 22237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:04.350232 22237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "a17f76f3390e4048af7ec7f431a93c78"
format_stamp: "Formatted at 2026-08-12 02:43:04 on dist-test-slave-ss4n"
server_key: "338e8c17ed96814cb4b2ae371c105b48"
server_key_iv: "2fa85e0cc963a04b4b70dbe3fe75b244"
server_key_version: "encryptionkey@0"
I20260812 02:43:04.351141 22237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "a17f76f3390e4048af7ec7f431a93c78"
format_stamp: "Formatted at 2026-08-12 02:43:04 on dist-test-slave-ss4n"
server_key: "338e8c17ed96814cb4b2ae371c105b48"
server_key_iv: "2fa85e0cc963a04b4b70dbe3fe75b244"
server_key_version: "encryptionkey@0"
I20260812 02:43:04.356988 22237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:43:04.361192 22254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:04.362902 22237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:43:04.363206 22237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "a17f76f3390e4048af7ec7f431a93c78"
format_stamp: "Formatted at 2026-08-12 02:43:04 on dist-test-slave-ss4n"
server_key: "338e8c17ed96814cb4b2ae371c105b48"
server_key_iv: "2fa85e0cc963a04b4b70dbe3fe75b244"
server_key_version: "encryptionkey@0"
I20260812 02:43:04.363442 22237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:04.386072 22237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:04.391170 22237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:04.391546 22237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:04.392802 22237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:04.394572 22237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:04.394698 22237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:04.394886 22237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:04.394980 22237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:04.435655 22237 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:45555
I20260812 02:43:04.435761 22374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:45555 every 8 connection(s)
I20260812 02:43:04.437161 22237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:04.440861 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 22237
I20260812 02:43:04.441113 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:43:04.441984 18195 external_mini_cluster.cc:1428] Setting key 19a4a63dc7bcab669e98841d363a7162
Aug 12 02:43:04 dist-test-slave-ss4n krb5kdc[22118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502584, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:04.462863 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:04.440321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:41095 (local address 127.17.196.254:46457)
0812 02:43:04.440913 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:04.440921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:04.442201 (+ 1280us) server_negotiation.cc:407] Connection header received
0812 02:43:04.444767 (+ 2566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:04.444774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:04.444866 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:04.445045 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:04.446862 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:04.448193 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:04.450097 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:04.450899 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:04.451077 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:04.455468 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:04.455474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:04.455478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:04.455521 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:04.458636 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:04.459790 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:04.459795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:04.459798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:04.459930 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:04.460620 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:04.460625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:04.460628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:04.461001 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:04.461140 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:04.462362 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0812 02:43:04.462597 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":283,"threads_started":1}
I20260812 02:43:04.464651 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:04.440891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46457 (local address 127.17.196.193:41095)
0812 02:43:04.441981 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:04.442035 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:04.444467 (+ 2432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:04.445296 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:04.445306 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:04.445972 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:04.446640 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:04.446667 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:04.448400 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:04.448405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:04.449914 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:04.449929 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:04.451166 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:04.451172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:04.451398 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:04.452208 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:04.452233 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:04.455172 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:04.458819 (+ 3647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:04.458834 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:04.458859 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:04.459634 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:04.460061 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:04.460070 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:04.460077 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:04.460435 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:04.461166 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:04.461179 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:04.461785 (+ 606us) client_negotiation.cc:769] Sending connection context
0812 02:43:04.462212 (+ 427us) client_negotiation.cc:240] Negotiation successful
0812 02:43:04.462913 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":464,"threads_started":1}
I20260812 02:43:04.466681 22375 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46457
I20260812 02:43:04.467085 22375 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:04.468072 22375 heartbeater.cc:511] Master 127.17.196.254:46457 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
I20260812 02:43:04.470755 22169 ts_manager.cc:194] Registered new tserver with Master: a17f76f3390e4048af7ec7f431a93c78 (127.17.196.193:45555)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
I20260812 02:43:04.472826 22169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:41095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:43:04.502686 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46457
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:04.860469 22382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:04.860791 22382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:04.860890 22382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:04.868705 22382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:04.868835 22382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:04.868955 22382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:04.879329 22382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:04.880743 22382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:04.882277 22382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:04.898171 22387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:04.898392 22390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:04.898382 22388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:04.900840 22382 server_base.cc:1034] running on GCE node
I20260812 02:43:04.901728 22382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:04.902894 22382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:04.904431 22382 hybrid_clock.cc:648] HybridClock initialized: now 1786502584904089 us; error 73 us; skew 500 ppm
I20260812 02:43:04.905242 22382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:04 dist-test-slave-ss4n krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502584, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:04.909380 22382 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:43:04.911644 22382 webserver.cc:460] Webserver started at http://127.17.196.194:44423/ using document root <none> and password file <none>
I20260812 02:43:04.912505 22382 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:04.912691 22382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:04.913115 22382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:04.917083 22382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "1fca6702c0854ff5a90289482501c045"
format_stamp: "Formatted at 2026-08-12 02:43:04 on dist-test-slave-ss4n"
server_key: "10355ec1b2b57d8ae88b3f6394c90b95"
server_key_iv: "23d4d014de7927f7936a43bc09c8040f"
server_key_version: "encryptionkey@0"
I20260812 02:43:04.918162 22382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "1fca6702c0854ff5a90289482501c045"
format_stamp: "Formatted at 2026-08-12 02:43:04 on dist-test-slave-ss4n"
server_key: "10355ec1b2b57d8ae88b3f6394c90b95"
server_key_iv: "23d4d014de7927f7936a43bc09c8040f"
server_key_version: "encryptionkey@0"
I20260812 02:43:04.926774 22382 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260812 02:43:04.932072 22399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:04.934247 22382 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 02:43:04.934521 22382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "1fca6702c0854ff5a90289482501c045"
format_stamp: "Formatted at 2026-08-12 02:43:04 on dist-test-slave-ss4n"
server_key: "10355ec1b2b57d8ae88b3f6394c90b95"
server_key_iv: "23d4d014de7927f7936a43bc09c8040f"
server_key_version: "encryptionkey@0"
I20260812 02:43:04.934803 22382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:04.952297 22382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:04.958051 22382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:04.958451 22382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:04.959820 22382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:04.961596 22382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:04.961714 22382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:04.961881 22382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:04.961961 22382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:05.000739 22382 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:36413
I20260812 02:43:05.000768 22517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:36413 every 8 connection(s)
I20260812 02:43:05.002830 22382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:05.009801 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 22382
I20260812 02:43:05.010074 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:43:05.010516 18195 external_mini_cluster.cc:1428] Setting key 3a1f74eb989f57a0c2a11549bee321bf
Aug 12 02:43:05 dist-test-slave-ss4n krb5kdc[22118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502584, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:05.035710 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.006438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:45365 (local address 127.17.196.254:46457)
0812 02:43:05.006990 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:05.006998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:05.008860 (+ 1862us) server_negotiation.cc:407] Connection header received
0812 02:43:05.012406 (+ 3546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:05.012418 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:05.012539 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:05.012639 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:05.015630 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.016802 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.020026 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.020833 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.020995 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:05.025893 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:05.025904 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:05.025912 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:05.025966 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:05.030231 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.031468 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.031477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.031484 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.031708 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.032426 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.032435 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.032440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.032910 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:05.033102 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:05.034545 (+ 1443us) server_negotiation.cc:299] Negotiation successful
0812 02:43:05.035006 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":236,"threads_started":1}
I20260812 02:43:05.036623 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.007182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46457 (local address 127.17.196.194:45365)
0812 02:43:05.008675 (+ 1493us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:05.008727 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:05.011970 (+ 3243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:05.013305 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:05.013323 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:05.014287 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:05.015353 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:05.015388 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.017033 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.017040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:05.019800 (+ 2760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:05.019824 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.021120 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.021129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:05.021549 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:05.022493 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:05.022532 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:05.025527 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:05.030502 (+ 4975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:05.030517 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:05.030540 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:05.031296 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:05.031886 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:05.031895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:05.031901 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:05.032235 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:05.033230 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:05.033242 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:05.033876 (+ 634us) client_negotiation.cc:769] Sending connection context
0812 02:43:05.034282 (+ 406us) client_negotiation.cc:240] Negotiation successful
0812 02:43:05.034958 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1178,"thread_start_us":943,"threads_started":1}
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
I20260812 02:43:05.042817 22518 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46457
I20260812 02:43:05.043272 22518 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:05.044209 22518 heartbeater.cc:511] Master 127.17.196.254:46457 requested a full tablet report, sending...
I20260812 02:43:05.046130 22169 ts_manager.cc:194] Registered new tserver with Master: 1fca6702c0854ff5a90289482501c045 (127.17.196.194:36413)
I20260812 02:43:05.047451 22169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:45365
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:43:05.077389 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46457
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:05.433599 22525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:05.433914 22525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:05.434013 22525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:05.441867 22525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:05.441993 22525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:05.442113 22525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:05.452425 22525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:05.453953 22525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:05.455358 22525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:05.470546 22530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:05.470619 22531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:05.470587 22533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:05.471383 22525 server_base.cc:1034] running on GCE node
I20260812 02:43:05.472143 22525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:05.473320 22525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:05.474563 22525 hybrid_clock.cc:648] HybridClock initialized: now 1786502585474531 us; error 88 us; skew 500 ppm
I20260812 02:43:05.474874 22525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:05.476428 22375 heartbeater.cc:503] Master 127.17.196.254:46457 was elected leader, sending a full tablet report...
Aug 12 02:43:05 dist-test-slave-ss4n krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502585, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:05.478868 22525 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:43:05.480508 22525 webserver.cc:460] Webserver started at http://127.17.196.195:36591/ using document root <none> and password file <none>
I20260812 02:43:05.481348 22525 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:05.481515 22525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:05.481956 22525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:05.485530 22525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "a2b9f7712dc540d984bf82bcd32c5517"
format_stamp: "Formatted at 2026-08-12 02:43:05 on dist-test-slave-ss4n"
server_key: "b1b3442347e3614028ce4cec30431b82"
server_key_iv: "9d2e1db58783f4d74893aec335d1c4a4"
server_key_version: "encryptionkey@0"
I20260812 02:43:05.486462 22525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "a2b9f7712dc540d984bf82bcd32c5517"
format_stamp: "Formatted at 2026-08-12 02:43:05 on dist-test-slave-ss4n"
server_key: "b1b3442347e3614028ce4cec30431b82"
server_key_iv: "9d2e1db58783f4d74893aec335d1c4a4"
server_key_version: "encryptionkey@0"
I20260812 02:43:05.492702 22525 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:43:05.496840 22543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:05.498347 22525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:43:05.498548 22525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "a2b9f7712dc540d984bf82bcd32c5517"
format_stamp: "Formatted at 2026-08-12 02:43:05 on dist-test-slave-ss4n"
server_key: "b1b3442347e3614028ce4cec30431b82"
server_key_iv: "9d2e1db58783f4d74893aec335d1c4a4"
server_key_version: "encryptionkey@0"
I20260812 02:43:05.498785 22525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:05.556569 22525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:05.561913 22525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:05.562280 22525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:05.563395 22525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:05.565033 22525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:05.565158 22525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:05.565328 22525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:05.565412 22525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:05.604647 22525 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:41895
I20260812 02:43:05.604693 22669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:41895 every 8 connection(s)
I20260812 02:43:05.606272 22525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:05.616274 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 22525
I20260812 02:43:05.616520 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:43:05.617072 18195 external_mini_cluster.cc:1428] Setting key 9b996e096dc94b6a02e466c61a6931a8
Aug 12 02:43:05 dist-test-slave-ss4n krb5kdc[22118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502585, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:05.630966 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.609215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:35051 (local address 127.17.196.254:46457)
0812 02:43:05.609772 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:05.609779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:05.611049 (+ 1270us) server_negotiation.cc:407] Connection header received
0812 02:43:05.613352 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:05.613357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:05.613450 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:05.613648 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:05.615336 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.616625 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.618929 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.619550 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.619731 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:05.624202 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:05.624208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:05.624212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:05.624255 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:05.627416 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.628276 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.628281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.628283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.628430 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.628927 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.628931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.628933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.629224 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:05.629357 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:05.630418 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0812 02:43:05.630662 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":287,"threads_started":1}
I20260812 02:43:05.632298 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.609924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46457 (local address 127.17.196.195:35051)
0812 02:43:05.610762 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:05.610854 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:05.613030 (+ 2176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:05.613867 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:05.613877 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:05.614459 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:05.615118 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:05.615139 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.616841 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.616847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:05.618720 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:05.618742 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.619850 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.619860 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:05.620157 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:05.621037 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:05.621098 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:05.623892 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:05.627600 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:05.627609 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:05.627625 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:05.628130 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:05.628573 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:05.628577 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:05.628579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:05.628792 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:05.629359 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:05.629366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:05.630031 (+ 665us) client_negotiation.cc:769] Sending connection context
0812 02:43:05.630303 (+ 272us) client_negotiation.cc:240] Negotiation successful
0812 02:43:05.630756 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":316,"threads_started":1}
I20260812 02:43:05.634389 22670 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46457
I20260812 02:43:05.634740 22670 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:05.635545 22670 heartbeater.cc:511] Master 127.17.196.254:46457 requested a full tablet report, sending...
I20260812 02:43:05.637194 22167 ts_manager.cc:194] Registered new tserver with Master: a2b9f7712dc540d984bf82bcd32c5517 (127.17.196.195:41895)
I20260812 02:43:05.638296 22167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:35051
I20260812 02:43:05.647012 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:43:05 dist-test-slave-ss4n krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:43:05 dist-test-slave-ss4n krb5kdc[22118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:43:05.712109 22679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.692746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51252 (local address 127.17.196.193:45555)
0812 02:43:05.693707 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:05.693716 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:05.693750 (+ 34us) server_negotiation.cc:407] Connection header received
0812 02:43:05.693828 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:05.693836 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:05.694070 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:05.694236 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:05.696123 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.697413 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.698958 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.699686 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.699868 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:05.704734 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:05.704751 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:05.704768 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:05.704821 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:05.708494 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.709284 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.709291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.709294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.709558 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.710185 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.710189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.710192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.710670 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:05.710949 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:05.711505 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 02:43:05.711741 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":564,"threads_started":1}
Aug 12 02:43:05 dist-test-slave-ss4n krb5kdc[22118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:05.733757 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.717374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.17.196.254:46457)
0812 02:43:05.717668 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:05.717675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:05.717842 (+ 167us) server_negotiation.cc:407] Connection header received
0812 02:43:05.717997 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:05.718025 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:05.718102 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:05.718196 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:05.719697 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.720529 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.722095 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.722588 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.722707 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:05.727371 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:05.727377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:05.727381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:05.727442 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:05.730735 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.731491 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.731498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.731501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.731681 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.732089 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.732095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.732098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.732464 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:05.732733 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:05.733088 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 02:43:05.733341 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
W20260812 02:43:05.734359 22167 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45746
I20260812 02:43:05.735527 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 22237
I20260812 02:43:05.750959 22679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.736501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.17.196.193:45555)
0812 02:43:05.736781 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:05.736788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:05.737062 (+ 274us) server_negotiation.cc:407] Connection header received
0812 02:43:05.737281 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:05.737288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:05.737401 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:05.737670 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:05.739066 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.739927 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.741392 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.741933 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.742099 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:05.744734 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:05.744739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:05.744743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:05.744781 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:05.747849 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.748650 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.748655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.748658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.748857 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.749302 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.749308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.749312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.749699 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:05.749971 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:05.750332 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 02:43:05.750593 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 02:43:05.751647 22370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:05.838469 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 22237
I20260812 02:43:05.859259 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 22382
Aug 12 02:43:05 dist-test-slave-ss4n krb5kdc[22118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:43:05.877951 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.860307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.17.196.194:36413)
0812 02:43:05.860870 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:05.860878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:05.860906 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:43:05.860968 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:05.860975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:05.861208 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:05.861342 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:05.862717 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.863815 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.865148 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.866002 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.866131 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:05.871017 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:05.871034 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:05.871053 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:05.871096 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:05.874432 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.875296 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.875305 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.875308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.875507 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:05.875955 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:05.875960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:05.875963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:05.876392 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:05.876721 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:05.877138 (+ 417us) server_negotiation.cc:299] Negotiation successful
0812 02:43:05.877570 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":308,"threads_started":1}
I20260812 02:43:05.879390 22513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:05.969767 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 22382
I20260812 02:43:05.992158 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 22525
Aug 12 02:43:05 dist-test-slave-ss4n krb5kdc[22118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:43:06.012215 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:05.993348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.17.196.195:41895)
0812 02:43:05.994122 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:05.994130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:05.994160 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:43:05.994224 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:05.994230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:05.994443 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:05.994600 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:05.996136 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.997171 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.998772 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:05.999345 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:05.999446 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:06.004494 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:06.004513 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:06.004534 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:06.004590 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:06.008476 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:06.009423 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:06.009432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:06.009435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:06.009714 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:06.010185 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:06.010189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:06.010192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:06.010720 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:06.011111 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:06.011547 (+ 436us) server_negotiation.cc:299] Negotiation successful
0812 02:43:06.011854 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":449,"threads_started":1}
I20260812 02:43:06.013823 22665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:06.100175 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 22525
I20260812 02:43:06.121953 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 22133
I20260812 02:43:06.123304 22205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:06.232232 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 22133
2026-08-12T02:43:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4400 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:43:06 dist-test-slave-ss4n krb5kdc[22693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:43:06 dist-test-slave-ss4n krb5kdc[22693](info): set up 2 sockets
Aug 12 02:43:06 dist-test-slave-ss4n krb5kdc[22693](info): commencing operation
krb5kdc: starting...
W20260812 02:43:07.489959 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:43:07 dist-test-slave-ss4n krb5kdc[22693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:43:07.750795 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39973
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:39973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:08.099778 22707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:08.100147 22707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:08.100327 22707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:08.107714 22707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:08.107827 22707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:08.107918 22707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:08.108014 22707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:08.108112 22707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:08.118546 22707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:39973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:08.119997 22707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:08.121430 22707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:08.133982 22716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:08.134053 22713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:08.134223 22714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:08.135067 22707 server_base.cc:1034] running on GCE node
I20260812 02:43:08.135911 22707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:08.137414 22707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:08.138679 22707 hybrid_clock.cc:648] HybridClock initialized: now 1786502588138628 us; error 51 us; skew 500 ppm
I20260812 02:43:08.139055 22707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:08 dist-test-slave-ss4n krb5kdc[22693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502588, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:08.142673 22707 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:43:08.144093 22707 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:08.144238 22707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:08.144645 22707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:08.147921 22707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "8bdbd89e28b94be89d7933cb8aa97a94"
format_stamp: "Formatted at 2026-08-12 02:43:08 on dist-test-slave-ss4n"
server_key: "cc07f32129605706271f7c4140bcf251"
server_key_iv: "7088f69d7fd1ad8d18116ab8ed25f8cb"
server_key_version: "encryptionkey@0"
I20260812 02:43:08.148785 22707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "8bdbd89e28b94be89d7933cb8aa97a94"
format_stamp: "Formatted at 2026-08-12 02:43:08 on dist-test-slave-ss4n"
server_key: "cc07f32129605706271f7c4140bcf251"
server_key_iv: "7088f69d7fd1ad8d18116ab8ed25f8cb"
server_key_version: "encryptionkey@0"
I20260812 02:43:08.154440 22707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:43:08.158318 22722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:08.159755 22707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:43:08.159963 22707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "8bdbd89e28b94be89d7933cb8aa97a94"
format_stamp: "Formatted at 2026-08-12 02:43:08 on dist-test-slave-ss4n"
server_key: "cc07f32129605706271f7c4140bcf251"
server_key_iv: "7088f69d7fd1ad8d18116ab8ed25f8cb"
server_key_version: "encryptionkey@0"
I20260812 02:43:08.160208 22707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:08.179505 22707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:08.184204 22707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:08.184511 22707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:08.206506 22707 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:39973
I20260812 02:43:08.206533 22779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:39973 every 8 connection(s)
I20260812 02:43:08.207698 22707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:08.211793 22780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:08.217993 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 22707
I20260812 02:43:08.218322 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:08.219056 18195 external_mini_cluster.cc:1428] Setting key e62dd90b034a7d2c0d35566b6a96d87b
I20260812 02:43:08.224426 22780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94: Bootstrap starting.
I20260812 02:43:08.228215 22780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:08.229621 22780 log.cc:824] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:08.233184 22780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94: No bootstrap required, opened a new log
Aug 12 02:43:08 dist-test-slave-ss4n krb5kdc[22693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:08.242833 22780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdbd89e28b94be89d7933cb8aa97a94" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39973 } }
I20260812 02:43:08.243340 22780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:08.243508 22780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bdbd89e28b94be89d7933cb8aa97a94, State: Initialized, Role: FOLLOWER
I20260812 02:43:08.244184 22780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdbd89e28b94be89d7933cb8aa97a94" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39973 } }
I20260812 02:43:08.244483 22780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:08.244659 22780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:08.244872 22780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:08.245072 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:08.221572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.17.196.254:39973)
0812 02:43:08.222591 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:08.222608 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:08.222657 (+ 49us) server_negotiation.cc:407] Connection header received
0812 02:43:08.224518 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:08.224552 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:08.225069 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:08.225533 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:08.227183 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:08.228664 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:08.230202 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:08.230973 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:08.231201 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:08.235566 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:08.235579 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:08.235596 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:08.235636 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:08.239454 (+ 3818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:08.240405 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:08.240414 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:08.240427 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:08.240688 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:08.241201 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:08.241206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:08.241208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:08.242010 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:08.242385 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:08.242938 (+ 553us) server_negotiation.cc:299] Negotiation successful
0812 02:43:08.243581 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":510,"threads_started":1}
I20260812 02:43:08.248257 22780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdbd89e28b94be89d7933cb8aa97a94" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39973 } }
I20260812 02:43:08.248903 22780 leader_election.cc:302] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bdbd89e28b94be89d7933cb8aa97a94; no voters:
I20260812 02:43:08.249691 22780 leader_election.cc:288] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:08.249853 22785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:08.250819 22785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [term 1 LEADER]: Becoming Leader. State: Replica: 8bdbd89e28b94be89d7933cb8aa97a94, State: Running, Role: LEADER
I20260812 02:43:08.251475 22785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdbd89e28b94be89d7933cb8aa97a94" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39973 } }
I20260812 02:43:08.252374 22780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:08.255000 22786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bdbd89e28b94be89d7933cb8aa97a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdbd89e28b94be89d7933cb8aa97a94" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39973 } } }
I20260812 02:43:08.255519 22787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bdbd89e28b94be89d7933cb8aa97a94. Latest consensus state: current_term: 1 leader_uuid: "8bdbd89e28b94be89d7933cb8aa97a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdbd89e28b94be89d7933cb8aa97a94" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39973 } } }
I20260812 02:43:08.256556 22786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:08.256680 22787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:08.262681 22795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:08.268735 22795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:08.278740 22795 catalog_manager.cc:1347] Generated new cluster ID: 34538c3cffaf4450ab7df0cb17f1b681
I20260812 02:43:08.278898 22795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:08.288699 22795 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:08.290319 22795 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:08.307935 22795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94: Generated new TSK 0
I20260812 02:43:08.309214 22795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:43:08.378931 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39973
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:08.729694 22808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:08.730057 22808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:08.730237 22808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:08.738895 22808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:08.739127 22808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:08.739360 22808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:08.750936 22808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:08.752641 22808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:08.754401 22808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:08.770632 22817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:08.770691 22814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:08.770735 22813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:08.771631 22808 server_base.cc:1034] running on GCE node
I20260812 02:43:08.772373 22808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:08.773785 22808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:08.775059 22808 hybrid_clock.cc:648] HybridClock initialized: now 1786502588774995 us; error 58 us; skew 500 ppm
I20260812 02:43:08.775507 22808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:08 dist-test-slave-ss4n krb5kdc[22693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502588, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:08.784253 22808 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:43:08.785774 22808 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:08.785920 22808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:08.786288 22808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:08.789463 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "f6e55e10d2ab4685b0d2a64388396b11"
format_stamp: "Formatted at 2026-08-12 02:43:08 on dist-test-slave-ss4n"
server_key: "eb2714a001c045a30b8738f50280e8b3"
server_key_iv: "5205f9670be96a81a9715b543bc47aa1"
server_key_version: "encryptionkey@0"
I20260812 02:43:08.790445 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "f6e55e10d2ab4685b0d2a64388396b11"
format_stamp: "Formatted at 2026-08-12 02:43:08 on dist-test-slave-ss4n"
server_key: "eb2714a001c045a30b8738f50280e8b3"
server_key_iv: "5205f9670be96a81a9715b543bc47aa1"
server_key_version: "encryptionkey@0"
I20260812 02:43:08.796453 22808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260812 02:43:08.800844 22823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:08.802567 22808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:43:08.802793 22808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "f6e55e10d2ab4685b0d2a64388396b11"
format_stamp: "Formatted at 2026-08-12 02:43:08 on dist-test-slave-ss4n"
server_key: "eb2714a001c045a30b8738f50280e8b3"
server_key_iv: "5205f9670be96a81a9715b543bc47aa1"
server_key_version: "encryptionkey@0"
I20260812 02:43:08.803066 22808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:08.847530 22808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:08.852708 22808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:08.853072 22808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:08.854358 22808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:08.855901 22808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:08.856050 22808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:08.856190 22808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:08.856287 22808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:08.897447 22808 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:40527
I20260812 02:43:08.897604 22946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:40527 every 8 connection(s)
I20260812 02:43:08.898770 22808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:08.908517 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 22808
I20260812 02:43:08.908802 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:43:08.909912 18195 external_mini_cluster.cc:1428] Setting key c10d3e8a2bea6f8921ad12df28aac299
Aug 12 02:43:08 dist-test-slave-ss4n krb5kdc[22693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502588, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:08.925441 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:08.901760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:44109 (local address 127.17.196.254:39973)
0812 02:43:08.902361 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:08.902369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:08.903548 (+ 1179us) server_negotiation.cc:407] Connection header received
0812 02:43:08.905582 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:08.905588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:08.905690 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:08.905805 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:08.908076 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:08.909112 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:08.911842 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:08.912556 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:08.912761 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:08.917742 (+ 4981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:08.917750 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:08.917754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:08.917800 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:08.920966 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:08.922420 (+ 1454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:08.922427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:08.922430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:08.922590 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:08.923203 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:08.923208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:08.923210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:08.923550 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:08.923739 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:08.924697 (+ 958us) server_negotiation.cc:299] Negotiation successful
0812 02:43:08.925016 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":441,"thread_start_us":281,"threads_started":1}
I20260812 02:43:08.926594 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:08.902515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39973 (local address 127.17.196.193:44109)
0812 02:43:08.903350 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:08.903400 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:08.905176 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:08.906168 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:08.906178 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:08.906990 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:08.907772 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:08.907796 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:08.909300 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:08.909307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:08.911617 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:08.911638 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:08.912899 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:08.912908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:08.913221 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:08.914113 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:08.914144 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:08.917350 (+ 3206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:08.921284 (+ 3934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:08.921300 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:08.921319 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:08.922209 (+ 890us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:08.922745 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:08.922751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:08.922756 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:08.923032 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:08.923711 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:08.923722 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:08.924155 (+ 433us) client_negotiation.cc:769] Sending connection context
0812 02:43:08.924550 (+ 395us) client_negotiation.cc:240] Negotiation successful
0812 02:43:08.925208 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":338,"threads_started":1}
I20260812 02:43:08.928766 22947 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39973
I20260812 02:43:08.929196 22947 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:08.930153 22947 heartbeater.cc:511] Master 127.17.196.254:39973 requested a full tablet report, sending...
I20260812 02:43:08.933037 22739 ts_manager.cc:194] Registered new tserver with Master: f6e55e10d2ab4685b0d2a64388396b11 (127.17.196.193:40527)
I20260812 02:43:08.935396 22739 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:44109
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:43:08.975172 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39973
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:09.334025 22954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:09.334331 22954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:09.334432 22954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:09.342105 22954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:09.342257 22954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:09.342375 22954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:09.352905 22954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:09.354476 22954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:09.355871 22954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:09.370703 22959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:09.370723 22960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:09.370752 22963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:09.371639 22954 server_base.cc:1034] running on GCE node
I20260812 02:43:09.372398 22954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:09.373669 22954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:09.374964 22954 hybrid_clock.cc:648] HybridClock initialized: now 1786502589374897 us; error 50 us; skew 500 ppm
I20260812 02:43:09.375442 22954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:09 dist-test-slave-ss4n krb5kdc[22693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502589, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:09.379391 22954 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:43:09.380861 22954 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:09.381006 22954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:09.381413 22954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:09.384697 22954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "1b2cfd8af17644479671059f3944aa44"
format_stamp: "Formatted at 2026-08-12 02:43:09 on dist-test-slave-ss4n"
server_key: "356e756d2c34ce5d1c09715a2acc4735"
server_key_iv: "f74f66582f642497c8c3fdf328c40aed"
server_key_version: "encryptionkey@0"
I20260812 02:43:09.385637 22954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "1b2cfd8af17644479671059f3944aa44"
format_stamp: "Formatted at 2026-08-12 02:43:09 on dist-test-slave-ss4n"
server_key: "356e756d2c34ce5d1c09715a2acc4735"
server_key_iv: "f74f66582f642497c8c3fdf328c40aed"
server_key_version: "encryptionkey@0"
I20260812 02:43:09.391194 22954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 02:43:09.394920 22969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:09.396595 22954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:43:09.396824 22954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "1b2cfd8af17644479671059f3944aa44"
format_stamp: "Formatted at 2026-08-12 02:43:09 on dist-test-slave-ss4n"
server_key: "356e756d2c34ce5d1c09715a2acc4735"
server_key_iv: "f74f66582f642497c8c3fdf328c40aed"
server_key_version: "encryptionkey@0"
I20260812 02:43:09.397143 22954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:09.437449 22954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:09.443043 22954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:09.443437 22954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:09.444589 22954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:09.446290 22954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:09.446431 22954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:09.446594 22954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:09.446702 22954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:09.485554 22954 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:45811
I20260812 02:43:09.485719 23097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:45811 every 8 connection(s)
I20260812 02:43:09.486501 22954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:09.493260 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 22954
I20260812 02:43:09.493561 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:43:09.494057 18195 external_mini_cluster.cc:1428] Setting key 1f445f47061ee47736235b7000e66d1f
Aug 12 02:43:09 dist-test-slave-ss4n krb5kdc[22693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502589, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:09.509106 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:09.489333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:36965 (local address 127.17.196.254:39973)
0812 02:43:09.489909 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:09.489914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:09.490991 (+ 1077us) server_negotiation.cc:407] Connection header received
0812 02:43:09.493094 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:09.493101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:09.493210 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:09.493347 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:09.495206 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:09.496102 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:09.497960 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:09.498452 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:09.498615 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:09.502884 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:09.502890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:09.502893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:09.502928 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:09.505812 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:09.506640 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:09.506645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:09.506647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:09.506783 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:09.507258 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:09.507262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:09.507264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:09.507554 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:09.507714 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:09.508468 (+ 754us) server_negotiation.cc:299] Negotiation successful
0812 02:43:09.508702 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":348,"threads_started":1}
I20260812 02:43:09.510259 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:09.489931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39973 (local address 127.17.196.194:36965)
0812 02:43:09.490776 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:09.490832 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:09.492759 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:09.493677 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:09.493692 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:09.494353 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:09.494984 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:09.495016 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:09.496273 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:09.496281 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:09.497798 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:09.497813 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:09.498717 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:09.498726 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:09.498980 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:09.499782 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:09.499808 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:09.502586 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:09.505992 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:09.506002 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:09.506021 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:09.506479 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:09.506903 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:09.506908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:09.506912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:09.507123 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:09.507690 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:09.507699 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:09.508075 (+ 376us) client_negotiation.cc:769] Sending connection context
0812 02:43:09.508371 (+ 296us) client_negotiation.cc:240] Negotiation successful
0812 02:43:09.508949 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":325,"threads_started":1}
I20260812 02:43:09.511941 23098 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39973
I20260812 02:43:09.512317 23098 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:09.513273 23098 heartbeater.cc:511] Master 127.17.196.254:39973 requested a full tablet report, sending...
I20260812 02:43:09.515443 22739 ts_manager.cc:194] Registered new tserver with Master: 1b2cfd8af17644479671059f3944aa44 (127.17.196.194:45811)
I20260812 02:43:09.516981 22739 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:36965
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:43:09.558455 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39973
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:09.911237 23105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:09.911603 23105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:09.911742 23105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:09.922549 23105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:09.922785 23105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:09.923185 23105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:09.936460 23105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:09.937961 23105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:09.939419 23105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:43:09.939409 22947 heartbeater.cc:503] Master 127.17.196.254:39973 was elected leader, sending a full tablet report...
W20260812 02:43:09.954857 23114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:09.954823 23111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:09.954957 23112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:09.955814 23105 server_base.cc:1034] running on GCE node
I20260812 02:43:09.956501 23105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:09.957772 23105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:09.959050 23105 hybrid_clock.cc:648] HybridClock initialized: now 1786502589958995 us; error 70 us; skew 500 ppm
I20260812 02:43:09.959435 23105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:09 dist-test-slave-ss4n krb5kdc[22693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502589, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:09.963004 23105 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:43:09.964512 23105 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:09.964659 23105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:09.965085 23105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:09.968461 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "5b472a2150e94b348de6bc5b8d8dcc2c"
format_stamp: "Formatted at 2026-08-12 02:43:09 on dist-test-slave-ss4n"
server_key: "e100efc518911322e9f72f93620448c1"
server_key_iv: "2db09ed89ad1626184162818b3758bba"
server_key_version: "encryptionkey@0"
I20260812 02:43:09.969353 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "5b472a2150e94b348de6bc5b8d8dcc2c"
format_stamp: "Formatted at 2026-08-12 02:43:09 on dist-test-slave-ss4n"
server_key: "e100efc518911322e9f72f93620448c1"
server_key_iv: "2db09ed89ad1626184162818b3758bba"
server_key_version: "encryptionkey@0"
I20260812 02:43:09.975126 23105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:43:09.978892 23122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:09.980383 23105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:43:09.980583 23105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "5b472a2150e94b348de6bc5b8d8dcc2c"
format_stamp: "Formatted at 2026-08-12 02:43:09 on dist-test-slave-ss4n"
server_key: "e100efc518911322e9f72f93620448c1"
server_key_iv: "2db09ed89ad1626184162818b3758bba"
server_key_version: "encryptionkey@0"
I20260812 02:43:09.980857 23105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:10.011529 23105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:10.016837 23105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:10.017163 23105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:10.018347 23105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:10.019824 23105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:10.019915 23105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:10.020056 23105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:10.020157 23105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:10.055665 23105 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:35019
I20260812 02:43:10.055759 23243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:35019 every 8 connection(s)
I20260812 02:43:10.056617 23105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:10.066499 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 23105
I20260812 02:43:10.066766 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:43:10.067267 18195 external_mini_cluster.cc:1428] Setting key cb2ac5ef32bb3908c3dd05b9482e62eb
Aug 12 02:43:10 dist-test-slave-ss4n krb5kdc[22693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502589, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:10.080046 23246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.059594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:60731 (local address 127.17.196.254:39973)
0812 02:43:10.060097 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.060102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.060927 (+ 825us) server_negotiation.cc:407] Connection header received
0812 02:43:10.062916 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.062922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.063023 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.063148 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:10.065232 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.066347 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.068747 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.069531 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.069672 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.073866 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:10.073871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:10.073874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:10.073911 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:10.076773 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.077759 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.077763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.077765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.077895 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.078358 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.078361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.078363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.078671 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:10.078791 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.079592 (+ 801us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.079767 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":267,"threads_started":1}
I20260812 02:43:10.081183 23247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.060033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39973 (local address 127.17.196.195:60731)
0812 02:43:10.060754 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.060800 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:10.062619 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:10.063403 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:10.063413 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:10.064266 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:10.064987 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.065008 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.066599 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.066604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.068440 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.068461 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.069798 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.069805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.070071 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.070979 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:10.071016 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:10.073602 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:10.076952 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:10.076961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:10.076975 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:10.077632 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:10.078029 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:10.078033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:10.078036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:10.078239 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:10.078803 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:10.078810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:10.079185 (+ 375us) client_negotiation.cc:769] Sending connection context
0812 02:43:10.079474 (+ 289us) client_negotiation.cc:240] Negotiation successful
0812 02:43:10.079905 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":279,"threads_started":1}
I20260812 02:43:10.082938 23244 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39973
I20260812 02:43:10.083277 23244 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:10.083976 23244 heartbeater.cc:511] Master 127.17.196.254:39973 requested a full tablet report, sending...
I20260812 02:43:10.085711 22739 ts_manager.cc:194] Registered new tserver with Master: 5b472a2150e94b348de6bc5b8d8dcc2c (127.17.196.195:35019)
I20260812 02:43:10.086637 22739 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:60731
I20260812 02:43:10.096604 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:43:10.115837 23246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.101451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.17.196.254:39973)
0812 02:43:10.101711 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.101719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.102277 (+ 558us) server_negotiation.cc:407] Connection header received
0812 02:43:10.102623 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.102628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.102698 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.102782 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:10.104319 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.105162 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.106788 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.107346 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.107455 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.110420 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:10.110426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:10.110429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:10.110465 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:10.113192 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.113996 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.114009 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.114011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.114151 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.114520 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.114525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.114528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.114870 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:10.115091 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.115419 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.115603 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 02:43:10.120682 22739 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:43:10.128186 22739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:10.156625 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.143721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:45811 (local address 127.0.0.1:40586)
0812 02:43:10.145354 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.145410 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:10.145732 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:10.146868 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:10.146880 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:10.146922 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:10.147529 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.147549 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.150154 (+ 2605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.150163 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.153511 (+ 3348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.153528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.155450 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.155456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.155697 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.155759 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 02:43:10.155907 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 02:43:10.156146 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1420,"mutex_wait_us":51,"spinlock_wait_cycles":35840,"thread_start_us":1243,"threads_started":1}
I20260812 02:43:10.156977 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.142953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.17.196.193:40527)
0812 02:43:10.143978 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.143988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.145542 (+ 1554us) server_negotiation.cc:407] Connection header received
0812 02:43:10.146002 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.146015 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.146326 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.146540 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:10.147971 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.149797 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.154021 (+ 4224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.155601 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.155816 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.156011 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.156292 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.156477 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":652,"threads_started":1}
I20260812 02:43:10.157823 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.142512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60004)
0812 02:43:10.145355 (+ 2843us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.145410 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:10.145711 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:10.147174 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:10.147178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:10.147208 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:10.147761 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.147775 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.150103 (+ 2328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.150111 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.153824 (+ 3713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.153839 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.155833 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.155840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.156047 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.156086 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:43:10.157180 (+ 1094us) client_negotiation.cc:240] Negotiation successful
0812 02:43:10.157332 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2589,"mutex_wait_us":54,"spinlock_wait_cycles":768,"thread_start_us":2332,"threads_started":1}
I20260812 02:43:10.158134 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.143194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.17.196.194:45811)
0812 02:43:10.144467 (+ 1273us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.144476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.145546 (+ 1070us) server_negotiation.cc:407] Connection header received
0812 02:43:10.146020 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.146028 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.146283 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.146516 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:10.147803 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.149919 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.153686 (+ 3767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.155226 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.155407 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.155605 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.157542 (+ 1937us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.157715 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":958,"threads_started":1}
I20260812 02:43:10.160107 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.145815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:35019 (local address 127.0.0.1:35942)
0812 02:43:10.151719 (+ 5904us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.151741 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:10.151893 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:10.152931 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:10.152936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:10.152966 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:10.153504 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.153525 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.155089 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.155095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.157411 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.157429 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.159324 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.159330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.159499 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.159532 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:43:10.159621 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 02:43:10.159757 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5764,"thread_start_us":5594,"threads_started":1}
I20260812 02:43:10.160236 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.146127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.17.196.195:35019)
0812 02:43:10.151351 (+ 5224us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.151358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.152064 (+ 706us) server_negotiation.cc:407] Connection header received
0812 02:43:10.152177 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.152186 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.152478 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.152699 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:10.153785 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.154932 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.157603 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.159135 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.159340 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.159522 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.159698 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.159880 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5044,"thread_start_us":4869,"threads_started":1}
I20260812 02:43:10.162506 22872 tablet_service.cc:1467] Processing CreateTablet for tablet c030188d5e5d497abf6c151963c2d891 (DEFAULT_TABLE table=test-table [id=77ec9ba1fde24683963db22ddd10bffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:10.162456 23022 tablet_service.cc:1467] Processing CreateTablet for tablet c030188d5e5d497abf6c151963c2d891 (DEFAULT_TABLE table=test-table [id=77ec9ba1fde24683963db22ddd10bffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:10.164137 22872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c030188d5e5d497abf6c151963c2d891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:10.164157 23022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c030188d5e5d497abf6c151963c2d891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:10.165630 23172 tablet_service.cc:1467] Processing CreateTablet for tablet c030188d5e5d497abf6c151963c2d891 (DEFAULT_TABLE table=test-table [id=77ec9ba1fde24683963db22ddd10bffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:10.166961 23172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c030188d5e5d497abf6c151963c2d891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:10.179812 23266 tablet_bootstrap.cc:492] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Bootstrap starting.
I20260812 02:43:10.184113 23266 tablet_bootstrap.cc:654] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:10.186332 23268 tablet_bootstrap.cc:492] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: Bootstrap starting.
I20260812 02:43:10.185374 23267 tablet_bootstrap.cc:492] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Bootstrap starting.
I20260812 02:43:10.187562 23266 log.cc:824] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:10.189889 23267 tablet_bootstrap.cc:654] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:10.190236 23268 tablet_bootstrap.cc:654] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:10.191005 23267 log.cc:824] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:10.191574 23266 tablet_bootstrap.cc:492] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: No bootstrap required, opened a new log
I20260812 02:43:10.191915 23266 ts_tablet_manager.cc:1397] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260812 02:43:10.191926 23268 log.cc:824] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:10.194152 23267 tablet_bootstrap.cc:492] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: No bootstrap required, opened a new log
I20260812 02:43:10.194427 23267 ts_tablet_manager.cc:1397] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260812 02:43:10.195240 23268 tablet_bootstrap.cc:492] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: No bootstrap required, opened a new log
I20260812 02:43:10.195531 23268 ts_tablet_manager.cc:1397] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 02:43:10.202994 23267 raft_consensus.cc:348] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.203498 23267 raft_consensus.cc:374] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:10.203608 23267 raft_consensus.cc:729] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2cfd8af17644479671059f3944aa44, State: Initialized, Role: FOLLOWER
I20260812 02:43:10.204335 23267 consensus_queue.cc:260] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.204783 23266 raft_consensus.cc:348] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.205297 23266 raft_consensus.cc:374] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:10.205425 23266 raft_consensus.cc:729] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b472a2150e94b348de6bc5b8d8dcc2c, State: Initialized, Role: FOLLOWER
I20260812 02:43:10.205796 23098 heartbeater.cc:503] Master 127.17.196.254:39973 was elected leader, sending a full tablet report...
I20260812 02:43:10.206210 23266 consensus_queue.cc:260] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.206444 23267 ts_tablet_manager.cc:1428] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260812 02:43:10.207603 23268 raft_consensus.cc:348] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.208045 23268 raft_consensus.cc:374] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:10.208143 23268 raft_consensus.cc:729] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e55e10d2ab4685b0d2a64388396b11, State: Initialized, Role: FOLLOWER
I20260812 02:43:10.208811 23268 consensus_queue.cc:260] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.210781 23268 ts_tablet_manager.cc:1428] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260812 02:43:10.211859 23244 heartbeater.cc:503] Master 127.17.196.254:39973 was elected leader, sending a full tablet report...
I20260812 02:43:10.212599 23266 ts_tablet_manager.cc:1428] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Time spent starting tablet: real 0.021s user 0.015s sys 0.004s
W20260812 02:43:10.238569 23099 tablet.cc:2369] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:10.290318 23275 raft_consensus.cc:482] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:10.290940 23275 raft_consensus.cc:504] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.293144 23275 leader_election.cc:288] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b2cfd8af17644479671059f3944aa44 (127.17.196.194:45811), 5b472a2150e94b348de6bc5b8d8dcc2c (127.17.196.195:35019)
I20260812 02:43:10.301321 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.293758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:46607 (local address 127.17.196.194:45811)
0812 02:43:10.294070 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.294078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.294483 (+ 405us) server_negotiation.cc:407] Connection header received
0812 02:43:10.294790 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.294799 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.294902 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.295007 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:10.295891 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.296881 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.299225 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.300253 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.300371 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.300423 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.300909 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.301024 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 02:43:10.301640 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.293650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:45811 (local address 127.17.196.193:46607)
0812 02:43:10.294341 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.294358 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:10.294550 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:10.295146 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:10.295150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:10.295178 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:10.295720 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.295736 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.297058 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.297065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.299029 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.299043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.300483 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.300491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.300697 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.300735 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 02:43:10.300875 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 02:43:10.301050 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"spinlock_wait_cycles":44416,"thread_start_us":405,"threads_started":1}
I20260812 02:43:10.302268 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.294573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:35019 (local address 127.17.196.193:56139)
0812 02:43:10.295264 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.295281 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:10.295405 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:10.295819 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:10.295825 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:10.295854 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:10.296256 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.296269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.297887 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.297890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.299676 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:10.299690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.301278 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.301285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:10.301468 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.301528 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 02:43:10.301692 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 02:43:10.301831 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":278,"threads_started":1}
I20260812 02:43:10.302771 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.294782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:56139 (local address 127.17.196.195:35019)
0812 02:43:10.295052 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.295061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.295376 (+ 315us) server_negotiation.cc:407] Connection header received
0812 02:43:10.295540 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.295546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.295635 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.295775 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:10.296522 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.297714 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.299839 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.301017 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.301180 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.301270 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.302353 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.302603 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 02:43:10.303085 23046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c030188d5e5d497abf6c151963c2d891" candidate_uuid: "f6e55e10d2ab4685b0d2a64388396b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b2cfd8af17644479671059f3944aa44" is_pre_election: true
I20260812 02:43:10.303771 23046 raft_consensus.cc:2454] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e55e10d2ab4685b0d2a64388396b11 in term 0.
I20260812 02:43:10.303695 23192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c030188d5e5d497abf6c151963c2d891" candidate_uuid: "f6e55e10d2ab4685b0d2a64388396b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" is_pre_election: true
I20260812 02:43:10.304270 23192 raft_consensus.cc:2454] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e55e10d2ab4685b0d2a64388396b11 in term 0.
I20260812 02:43:10.304585 22824 leader_election.cc:302] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b2cfd8af17644479671059f3944aa44, f6e55e10d2ab4685b0d2a64388396b11; no voters:
I20260812 02:43:10.305116 23275 raft_consensus.cc:2781] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:10.305305 23275 raft_consensus.cc:482] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:10.305428 23275 raft_consensus.cc:3037] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:10.307888 23275 raft_consensus.cc:504] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.308561 23275 leader_election.cc:288] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [CANDIDATE]: Term 1 election: Requested vote from peers 1b2cfd8af17644479671059f3944aa44 (127.17.196.194:45811), 5b472a2150e94b348de6bc5b8d8dcc2c (127.17.196.195:35019)
W20260812 02:43:10.308465 23245 tablet.cc:2369] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:10.309216 23046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c030188d5e5d497abf6c151963c2d891" candidate_uuid: "f6e55e10d2ab4685b0d2a64388396b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b2cfd8af17644479671059f3944aa44"
I20260812 02:43:10.309262 23192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c030188d5e5d497abf6c151963c2d891" candidate_uuid: "f6e55e10d2ab4685b0d2a64388396b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c"
I20260812 02:43:10.309500 23192 raft_consensus.cc:3037] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:10.309460 23046 raft_consensus.cc:3037] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:10.311820 23046 raft_consensus.cc:2454] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e55e10d2ab4685b0d2a64388396b11 in term 1.
I20260812 02:43:10.311838 23192 raft_consensus.cc:2454] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e55e10d2ab4685b0d2a64388396b11 in term 1.
I20260812 02:43:10.312340 22824 leader_election.cc:302] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b2cfd8af17644479671059f3944aa44, f6e55e10d2ab4685b0d2a64388396b11; no voters:
I20260812 02:43:10.312654 23275 raft_consensus.cc:2781] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:10.313179 23275 raft_consensus.cc:686] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 1 LEADER]: Becoming Leader. State: Replica: f6e55e10d2ab4685b0d2a64388396b11, State: Running, Role: LEADER
I20260812 02:43:10.313757 23275 consensus_queue.cc:237] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } }
I20260812 02:43:10.319211 22738 catalog_manager.cc:5495] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e55e10d2ab4685b0d2a64388396b11 (127.17.196.193). New cstate: current_term: 1 leader_uuid: "f6e55e10d2ab4685b0d2a64388396b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e55e10d2ab4685b0d2a64388396b11" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 } health_report { overall_health: UNKNOWN } } }
I20260812 02:43:10.358143 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.351977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.17.196.193:40527)
0812 02:43:10.352264 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.352269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.352290 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:43:10.352393 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.352397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.352484 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.352642 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:10.353149 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.354113 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.355700 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.356142 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.356252 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.356868 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:10.357441 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:10.357693 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.357829 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.357909 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260812 02:43:10.367708 23046 raft_consensus.cc:1261] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 1 FOLLOWER]: Refusing update from remote peer f6e55e10d2ab4685b0d2a64388396b11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:10.367708 23192 raft_consensus.cc:1261] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 1 FOLLOWER]: Refusing update from remote peer f6e55e10d2ab4685b0d2a64388396b11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:10.369035 23278 consensus_queue.cc:1035] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b2cfd8af17644479671059f3944aa44" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 45811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:10.369518 23275 consensus_queue.cc:1035] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b472a2150e94b348de6bc5b8d8dcc2c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 35019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:10.398828 22737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:10.399200 22737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:10.404466 22737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8bdbd89e28b94be89d7933cb8aa97a94: Sending DeleteTablet for 3 replicas of tablet c030188d5e5d497abf6c151963c2d891
I20260812 02:43:10.405907 22872 tablet_service.cc:1514] Processing DeleteTablet for tablet c030188d5e5d497abf6c151963c2d891 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:10 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:60004
I20260812 02:43:10.406064 23172 tablet_service.cc:1514] Processing DeleteTablet for tablet c030188d5e5d497abf6c151963c2d891 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:10 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:35942
I20260812 02:43:10.407137 23022 tablet_service.cc:1514] Processing DeleteTablet for tablet c030188d5e5d497abf6c151963c2d891 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:10 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:40586
I20260812 02:43:10.409044 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 22808
I20260812 02:43:10.431906 23294 tablet_replica.cc:330] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: stopping tablet replica
I20260812 02:43:10.432513 23294 raft_consensus.cc:2229] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:10.433168 23294 raft_consensus.cc:2258] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:10.436939 23294 ts_tablet_manager.cc:1905] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:10.438920 23292 tablet_replica.cc:330] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: stopping tablet replica
Aug 12 02:43:10 dist-test-slave-ss4n krb5kdc[22693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:43:10.439443 23292 raft_consensus.cc:2229] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:43:10.440086 23292 raft_consensus.cc:2258] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:10.440531 23293 tablet_replica.cc:330] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: stopping tablet replica
I20260812 02:43:10.443305 23292 ts_tablet_manager.cc:1905] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:10.446584 23294 ts_tablet_manager.cc:1918] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:10.446808 23294 log.cc:1196] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/c030188d5e5d497abf6c151963c2d891
I20260812 02:43:10.447466 23294 ts_tablet_manager.cc:1939] T c030188d5e5d497abf6c151963c2d891 P 1b2cfd8af17644479671059f3944aa44: Deleting consensus metadata
I20260812 02:43:10.449723 22723 catalog_manager.cc:4841] TS 1b2cfd8af17644479671059f3944aa44 (127.17.196.194:45811): tablet c030188d5e5d497abf6c151963c2d891 (table test-table [id=77ec9ba1fde24683963db22ddd10bffd]) successfully deleted
I20260812 02:43:10.450621 23293 raft_consensus.cc:2229] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:10.451095 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.411049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.17.196.193:40527)
0812 02:43:10.411359 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.411371 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.422319 (+ 10948us) server_negotiation.cc:407] Connection header received
0812 02:43:10.423024 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.423035 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.423166 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.423280 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:10.426017 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.427878 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.431303 (+ 3425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.432449 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.432618 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.440591 (+ 7973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:10.440610 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:10.440632 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:10.440690 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:10.445625 (+ 4935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.446815 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.446826 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.446830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.447156 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.447846 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.447855 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.447862 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.448467 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:10.448861 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.450379 (+ 1518us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.450677 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 02:43:10.451678 23293 raft_consensus.cc:2258] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:10.451941 22942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:10.454011 23293 ts_tablet_manager.cc:1905] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:10.460644 23293 ts_tablet_manager.cc:1918] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:10.460811 23293 log.cc:1196] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/c030188d5e5d497abf6c151963c2d891
I20260812 02:43:10.461355 23293 ts_tablet_manager.cc:1939] T c030188d5e5d497abf6c151963c2d891 P 5b472a2150e94b348de6bc5b8d8dcc2c: Deleting consensus metadata
I20260812 02:43:10.463157 22723 catalog_manager.cc:4841] TS 5b472a2150e94b348de6bc5b8d8dcc2c (127.17.196.195:35019): tablet c030188d5e5d497abf6c151963c2d891 (table test-table [id=77ec9ba1fde24683963db22ddd10bffd]) successfully deleted
I20260812 02:43:10.562228 23292 ts_tablet_manager.cc:1918] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:10.562569 23292 log.cc:1196] T c030188d5e5d497abf6c151963c2d891 P f6e55e10d2ab4685b0d2a64388396b11: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/c030188d5e5d497abf6c151963c2d891
I20260812 02:43:10.562954 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 22808
W20260812 02:43:10.587332 22726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.196.193:40527 (error 108)
I20260812 02:43:10.588272 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 22954
I20260812 02:43:10.589782 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.589160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60024)
0812 02:43:10.589407 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.589600 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260812 02:43:10.590086 22726 catalog_manager.cc:4584] TS f6e55e10d2ab4685b0d2a64388396b11 (127.17.196.193:40527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c030188d5e5d497abf6c151963c2d891: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Aug 12 02:43:10 dist-test-slave-ss4n krb5kdc[22693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:43:10.606710 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.589349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.17.196.194:45811)
0812 02:43:10.589669 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.589676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.589697 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:43:10.589861 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.589869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.589959 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.590045 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:10.591708 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.592524 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.594323 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.594954 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.595096 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.599739 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:10.599756 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:10.599772 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:10.599817 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:10.603342 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.604210 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.604219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.604222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.604386 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.604870 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.604874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.604876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.605319 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:10.605764 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.606126 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.606357 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260812 02:43:10.607592 23093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:10.644012 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.643441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60028)
0812 02:43:10.643743 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.643843 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 02:43:10.645162 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.644788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60034)
0812 02:43:10.644949 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.645029 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 02:43:10.701638 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 22954
I20260812 02:43:10.706013 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.705334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60042)
0812 02:43:10.705659 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.705788 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 02:43:10.707386 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.706868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60058)
0812 02:43:10.707079 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.707197 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 02:43:10.726025 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 23105
Aug 12 02:43:10 dist-test-slave-ss4n krb5kdc[22693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:43:10.742275 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.726949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35944 (local address 127.17.196.195:35019)
0812 02:43:10.727184 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:10.727189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:10.727211 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:43:10.727294 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:10.727299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:10.727380 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:10.727497 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:10.728829 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.729782 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.731080 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:10.731477 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:10.731588 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:10.736226 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:10.736244 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:10.736263 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:10.736305 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:10.739150 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.739945 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.739953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.739955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.740110 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:10.740529 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:10.740536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:10.740539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:10.740933 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:10.741275 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:10.741644 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 02:43:10.741882 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 02:43:10.743141 23239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:10.788699 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.788015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60062)
0812 02:43:10.788312 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.788442 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 02:43:10.790201 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.789674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60068)
0812 02:43:10.789908 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.790049 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 02:43:10.843695 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 23105
I20260812 02:43:10.867277 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 22707
I20260812 02:43:10.868723 22775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:10.983492 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:10.982630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40527 (local address 127.0.0.1:60080)
0812 02:43:10.983087 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:10.983217 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260812 02:43:10.983758 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 22707
2026-08-12T02:43:11Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4755 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T02:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:11Z Disabled control of system clock
I20260812 02:43:11.072932 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39305
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:40881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:39305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 02:43:11.423053 23309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:11.423388 23309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:11.423532 23309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:11.431208 23309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:11.431329 23309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:11.431370 23309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:11.431468 23309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:11.431540 23309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:11.441376 23309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:39305
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:11.442960 23309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:11.444926 23309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:11.458212 23314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:11.458184 23317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:11.458148 23315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:11.459017 23309 server_base.cc:1034] running on GCE node
I20260812 02:43:11.459862 23309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:11.461445 23309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:11.462735 23309 hybrid_clock.cc:648] HybridClock initialized: now 1786502591462679 us; error 53 us; skew 500 ppm
I20260812 02:43:11.463111 23309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:11.465750 23309 webserver.cc:460] Webserver started at http://127.17.196.254:38435/ using document root <none> and password file <none>
I20260812 02:43:11.466491 23309 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:11.466586 23309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:11.466984 23309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:11.470112 23309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "0961f26cf80f4756a0cfe99ed91cda77"
format_stamp: "Formatted at 2026-08-12 02:43:11 on dist-test-slave-ss4n"
server_key: "a53a9d1edd6ffb7c64db805e683aa531"
server_key_iv: "160f09251ad0d7787a180bc4bef4722b"
server_key_version: "encryptionkey@0"
I20260812 02:43:11.470965 23309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "0961f26cf80f4756a0cfe99ed91cda77"
format_stamp: "Formatted at 2026-08-12 02:43:11 on dist-test-slave-ss4n"
server_key: "a53a9d1edd6ffb7c64db805e683aa531"
server_key_iv: "160f09251ad0d7787a180bc4bef4722b"
server_key_version: "encryptionkey@0"
I20260812 02:43:11.476689 23309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:43:11.480528 23323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:11.481953 23309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:43:11.482153 23309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "0961f26cf80f4756a0cfe99ed91cda77"
format_stamp: "Formatted at 2026-08-12 02:43:11 on dist-test-slave-ss4n"
server_key: "a53a9d1edd6ffb7c64db805e683aa531"
server_key_iv: "160f09251ad0d7787a180bc4bef4722b"
server_key_version: "encryptionkey@0"
I20260812 02:43:11.482403 23309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:11.488989 23309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:11.489984 23309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:11.490347 23309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:11.512089 23309 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:39305
I20260812 02:43:11.512112 23379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:39305 every 8 connection(s)
I20260812 02:43:11.513705 23309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:11.517776 23380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:11.518414 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 23309
I20260812 02:43:11.518657 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:11.519378 18195 external_mini_cluster.cc:1428] Setting key 8f10b734f745d1564ef1aa7442108f1b
I20260812 02:43:11.529774 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:11.521697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.17.196.254:39305)
0812 02:43:11.522930 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:11.522942 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:11.523007 (+ 65us) server_negotiation.cc:407] Connection header received
0812 02:43:11.524946 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:11.524999 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:11.525749 (+ 750us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:11.526320 (+ 571us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:11.527476 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:11.527485 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:11.527505 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:11.527525 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:11.527614 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:11.527640 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:11.527894 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:11.528060 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 02:43:11.528384 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":419,"threads_started":1}
I20260812 02:43:11.534300 23380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77: Bootstrap starting.
I20260812 02:43:11.537572 23380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:11.538666 23380 log.cc:824] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:11.542426 23380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77: No bootstrap required, opened a new log
I20260812 02:43:11.551576 23380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0961f26cf80f4756a0cfe99ed91cda77" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39305 } }
I20260812 02:43:11.551987 23380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:11.552073 23380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0961f26cf80f4756a0cfe99ed91cda77, State: Initialized, Role: FOLLOWER
I20260812 02:43:11.552615 23380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0961f26cf80f4756a0cfe99ed91cda77" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39305 } }
I20260812 02:43:11.552883 23380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:11.552994 23380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:11.553129 23380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:11.555980 23380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0961f26cf80f4756a0cfe99ed91cda77" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39305 } }
I20260812 02:43:11.556488 23380 leader_election.cc:302] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0961f26cf80f4756a0cfe99ed91cda77; no voters:
I20260812 02:43:11.557194 23380 leader_election.cc:288] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:11.557339 23385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:11.558257 23385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [term 1 LEADER]: Becoming Leader. State: Replica: 0961f26cf80f4756a0cfe99ed91cda77, State: Running, Role: LEADER
I20260812 02:43:11.558859 23385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0961f26cf80f4756a0cfe99ed91cda77" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39305 } }
I20260812 02:43:11.559597 23380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:11.563129 23387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0961f26cf80f4756a0cfe99ed91cda77. Latest consensus state: current_term: 1 leader_uuid: "0961f26cf80f4756a0cfe99ed91cda77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0961f26cf80f4756a0cfe99ed91cda77" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39305 } } }
I20260812 02:43:11.563131 23386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0961f26cf80f4756a0cfe99ed91cda77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0961f26cf80f4756a0cfe99ed91cda77" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39305 } } }
I20260812 02:43:11.563700 23386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:11.563705 23387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:11.567355 23391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:11.573233 23391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:11.584591 23391 catalog_manager.cc:1347] Generated new cluster ID: 2b8195921e7f40fb905208ca3baf7dbc
I20260812 02:43:11.584722 23391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:11.604507 23391 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:11.605067 23391 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:11.614805 23391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77: Generated new TSK 0
I20260812 02:43:11.615803 23391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:43:11.621460 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39305
--builtin_ntp_servers=127.17.196.212:40881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 02:43:11.991894 23406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:11.992229 23406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:11.992329 23406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:12.000850 23406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:12.000974 23406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:12.001099 23406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:12.011834 23406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:12.013249 23406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:12.015273 23406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:12.030233 23412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:12.030478 23416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:12.030874 23413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:12.032189 23406 server_base.cc:1034] running on GCE node
I20260812 02:43:12.032876 23406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:12.034039 23406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:12.035295 23406 hybrid_clock.cc:648] HybridClock initialized: now 1786502592035241 us; error 56 us; skew 500 ppm
I20260812 02:43:12.035662 23406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:12.038496 23406 webserver.cc:460] Webserver started at http://127.17.196.193:33091/ using document root <none> and password file <none>
I20260812 02:43:12.039363 23406 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:12.039525 23406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:12.039920 23406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:12.043423 23406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "f91acc033b2c4cfdb9963f01715e110e"
format_stamp: "Formatted at 2026-08-12 02:43:12 on dist-test-slave-ss4n"
server_key: "3c1aae7a3995216f4b1e005800e977cd"
server_key_iv: "25eb3972cf93e107b5185a3bb5bf2571"
server_key_version: "encryptionkey@0"
I20260812 02:43:12.044430 23406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "f91acc033b2c4cfdb9963f01715e110e"
format_stamp: "Formatted at 2026-08-12 02:43:12 on dist-test-slave-ss4n"
server_key: "3c1aae7a3995216f4b1e005800e977cd"
server_key_iv: "25eb3972cf93e107b5185a3bb5bf2571"
server_key_version: "encryptionkey@0"
I20260812 02:43:12.050621 23406 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 02:43:12.054478 23422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:12.056118 23406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 02:43:12.056347 23406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "f91acc033b2c4cfdb9963f01715e110e"
format_stamp: "Formatted at 2026-08-12 02:43:12 on dist-test-slave-ss4n"
server_key: "3c1aae7a3995216f4b1e005800e977cd"
server_key_iv: "25eb3972cf93e107b5185a3bb5bf2571"
server_key_version: "encryptionkey@0"
I20260812 02:43:12.056643 23406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:12.063660 23406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:12.064654 23406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:12.064980 23406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:12.066202 23406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:12.068068 23406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:12.068186 23406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:12.068337 23406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:12.068447 23406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:12.105090 23406 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:42697
I20260812 02:43:12.105211 23547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:42697 every 8 connection(s)
I20260812 02:43:12.106719 23406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:12.108814 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 23406
I20260812 02:43:12.109061 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:43:12.109977 18195 external_mini_cluster.cc:1428] Setting key 1630845013bf0b4561342a722ac35de7
I20260812 02:43:12.113686 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39305
--builtin_ntp_servers=127.17.196.212:40881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 02:43:12.117031 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:12.109842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:48985 (local address 127.17.196.254:39305)
0812 02:43:12.110446 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:12.110451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:12.112104 (+ 1653us) server_negotiation.cc:407] Connection header received
0812 02:43:12.114344 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:12.114353 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:12.114433 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:12.114573 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:12.116097 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:12.116106 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:12.116113 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:12.116137 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:12.116215 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:12.116235 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:12.116375 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:12.116585 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 02:43:12.116657 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":323,"threads_started":1}
I20260812 02:43:12.118101 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:12.110626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39305 (local address 127.17.196.193:48985)
0812 02:43:12.111892 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:12.111964 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:12.113995 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:12.114849 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:12.114858 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:12.114982 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:12.115756 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:43:12.115759 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:12.115800 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:43:12.115895 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:12.116356 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:12.116365 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:12.116371 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:43:12.116623 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 02:43:12.116846 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":374,"threads_started":1}
I20260812 02:43:12.119879 23548 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39305
I20260812 02:43:12.120237 23548 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:12.120922 23548 heartbeater.cc:511] Master 127.17.196.254:39305 requested a full tablet report, sending...
I20260812 02:43:12.123262 23341 ts_manager.cc:194] Registered new tserver with Master: f91acc033b2c4cfdb9963f01715e110e (127.17.196.193:42697)
W20260812 02:43:12.467754 23553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:12.468098 23553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:12.468271 23553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:12.475883 23553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:12.476068 23553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:12.476270 23553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:12.486352 23553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:12.487886 23553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:12.489848 23553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:12.504740 23561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:12.504746 23560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:12.504787 23563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:12.510943 23553 server_base.cc:1034] running on GCE node
I20260812 02:43:12.511829 23553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:12.513219 23553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:12.514492 23553 hybrid_clock.cc:648] HybridClock initialized: now 1786502592514434 us; error 47 us; skew 500 ppm
I20260812 02:43:12.514912 23553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:12.518033 23553 webserver.cc:460] Webserver started at http://127.17.196.194:43637/ using document root <none> and password file <none>
I20260812 02:43:12.518973 23553 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:12.519095 23553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:12.519464 23553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:12.523675 23553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "b7ca86b7d9a24c539dda104282b604c9"
format_stamp: "Formatted at 2026-08-12 02:43:12 on dist-test-slave-ss4n"
server_key: "83384b8b47934e234cbf5bd1b738ca0b"
server_key_iv: "ea035f82cdfbc642ada6acc0a65c8023"
server_key_version: "encryptionkey@0"
I20260812 02:43:12.524742 23553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "b7ca86b7d9a24c539dda104282b604c9"
format_stamp: "Formatted at 2026-08-12 02:43:12 on dist-test-slave-ss4n"
server_key: "83384b8b47934e234cbf5bd1b738ca0b"
server_key_iv: "ea035f82cdfbc642ada6acc0a65c8023"
server_key_version: "encryptionkey@0"
I20260812 02:43:12.531824 23553 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260812 02:43:12.536350 23569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:12.538024 23553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:43:12.538177 23553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "b7ca86b7d9a24c539dda104282b604c9"
format_stamp: "Formatted at 2026-08-12 02:43:12 on dist-test-slave-ss4n"
server_key: "83384b8b47934e234cbf5bd1b738ca0b"
server_key_iv: "ea035f82cdfbc642ada6acc0a65c8023"
server_key_version: "encryptionkey@0"
I20260812 02:43:12.538363 23553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:12.544700 23553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:12.545742 23553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:12.546015 23553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:12.547039 23553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:12.548573 23553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:12.548667 23553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:12.548749 23553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:12.548821 23553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:12.586645 23553 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:36821
I20260812 02:43:12.586745 23692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:36821 every 8 connection(s)
I20260812 02:43:12.588085 23553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:12.593073 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 23553
I20260812 02:43:12.593264 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:43:12.593818 18195 external_mini_cluster.cc:1428] Setting key a91261a16db96409669571fb9d12e021
I20260812 02:43:12.596982 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39305
--builtin_ntp_servers=127.17.196.212:40881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 02:43:12.597886 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:12.591277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:54979 (local address 127.17.196.254:39305)
0812 02:43:12.591552 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:12.591559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:12.593070 (+ 1511us) server_negotiation.cc:407] Connection header received
0812 02:43:12.595282 (+ 2212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:12.595286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:12.595340 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:12.595430 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:12.597072 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:12.597078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:12.597083 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:12.597103 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:12.597147 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:12.597159 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:12.597308 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:12.597619 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 02:43:12.597676 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260812 02:43:12.599403 23695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:12.591819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39305 (local address 127.17.196.194:54979)
0812 02:43:12.592862 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:12.592910 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:12.594958 (+ 2048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:12.595728 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:12.595742 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:12.595901 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:12.596746 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:43:12.596748 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:12.596793 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:43:12.596882 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:12.597352 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:12.597366 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:12.597378 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 02:43:12.597704 (+ 326us) client_negotiation.cc:240] Negotiation successful
0812 02:43:12.598003 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":312,"threads_started":1}
I20260812 02:43:12.600991 23693 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39305
I20260812 02:43:12.601361 23693 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:12.602119 23693 heartbeater.cc:511] Master 127.17.196.254:39305 requested a full tablet report, sending...
I20260812 02:43:12.603869 23341 ts_manager.cc:194] Registered new tserver with Master: b7ca86b7d9a24c539dda104282b604c9 (127.17.196.194:36821)
W20260812 02:43:12.981421 23697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:12.981849 23697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:12.982031 23697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:12.990111 23697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:12.990269 23697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:12.990468 23697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:13.000908 23697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:13.002372 23697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:13.004370 23697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:13.019086 23702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:13.019273 23705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:13.020290 23707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:13.026444 23697 server_base.cc:1034] running on GCE node
I20260812 02:43:13.027387 23697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:13.028612 23697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:13.029925 23697 hybrid_clock.cc:648] HybridClock initialized: now 1786502593029843 us; error 57 us; skew 500 ppm
I20260812 02:43:13.030421 23697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:13.033603 23697 webserver.cc:460] Webserver started at http://127.17.196.195:38305/ using document root <none> and password file <none>
I20260812 02:43:13.034415 23697 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:13.034580 23697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:13.035005 23697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:13.038798 23697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "28ae2eb7695542ca80622352a5b0ca38"
format_stamp: "Formatted at 2026-08-12 02:43:13 on dist-test-slave-ss4n"
server_key: "6349385983b4278cd2e2a301f78512d5"
server_key_iv: "e369176e686066766a4f18f850cdf318"
server_key_version: "encryptionkey@0"
I20260812 02:43:13.039795 23697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "28ae2eb7695542ca80622352a5b0ca38"
format_stamp: "Formatted at 2026-08-12 02:43:13 on dist-test-slave-ss4n"
server_key: "6349385983b4278cd2e2a301f78512d5"
server_key_iv: "e369176e686066766a4f18f850cdf318"
server_key_version: "encryptionkey@0"
I20260812 02:43:13.046474 23697 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260812 02:43:13.050870 23714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:13.052907 23697 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260812 02:43:13.053177 23697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "28ae2eb7695542ca80622352a5b0ca38"
format_stamp: "Formatted at 2026-08-12 02:43:13 on dist-test-slave-ss4n"
server_key: "6349385983b4278cd2e2a301f78512d5"
server_key_iv: "e369176e686066766a4f18f850cdf318"
server_key_version: "encryptionkey@0"
I20260812 02:43:13.053537 23697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:13.060842 23697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:13.062198 23697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:13.062542 23697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:13.063617 23697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:13.065418 23697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:13.065580 23697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 02:43:13.065739 23697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:13.065864 23697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:13.117542 23697 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:36911
I20260812 02:43:13.117596 23848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:36911 every 8 connection(s)
I20260812 02:43:13.119057 23697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:13.126854 23548 heartbeater.cc:503] Master 127.17.196.254:39305 was elected leader, sending a full tablet report...
I20260812 02:43:13.127123 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 23697
I20260812 02:43:13.127303 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:43:13.127923 18195 external_mini_cluster.cc:1428] Setting key 49631273a99e0da6f8c8892bddaf38ff
I20260812 02:43:13.131026 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.122396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:58973 (local address 127.17.196.254:39305)
0812 02:43:13.122911 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.122915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.124357 (+ 1442us) server_negotiation.cc:407] Connection header received
0812 02:43:13.127356 (+ 2999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.127366 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.127447 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.127559 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.129661 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.129668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.129674 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.129704 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.129754 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.129768 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.129931 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.130470 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.130561 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":280,"threads_started":1}
I20260812 02:43:13.132382 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.122913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39305 (local address 127.17.196.195:58973)
0812 02:43:13.124118 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:13.124172 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:13.126864 (+ 2692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:13.128063 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:13.128077 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:13.128229 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:13.129195 (+ 966us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:43:13.129200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:13.129261 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:43:13.129400 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:13.129901 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:13.129915 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:13.129925 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 02:43:13.130545 (+ 620us) client_negotiation.cc:240] Negotiation successful
0812 02:43:13.130913 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":318,"threads_started":1}
I20260812 02:43:13.134179 23849 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39305
I20260812 02:43:13.134552 23849 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:13.135279 23849 heartbeater.cc:511] Master 127.17.196.254:39305 requested a full tablet report, sending...
I20260812 02:43:13.137178 23341 ts_manager.cc:194] Registered new tserver with Master: 28ae2eb7695542ca80622352a5b0ca38 (127.17.196.195:36911)
I20260812 02:43:13.145184 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:43:13.153357 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.150609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.17.196.254:39305)
0812 02:43:13.150805 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.150810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.151443 (+ 633us) server_negotiation.cc:407] Connection header received
0812 02:43:13.151595 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.151599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.151655 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.151748 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.152786 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.152790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.152793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.152810 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.152849 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.152862 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.152967 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.153094 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.153148 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:43:13.158131 23341 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:43:13.166946 23341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:13.187652 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.181032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.17.196.194:36821)
0812 02:43:13.183089 (+ 2057us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.183103 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.183145 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:43:13.183242 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.183254 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.183567 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.183752 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.186421 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.186430 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.186447 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.186466 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.186555 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.186578 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.186728 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.187128 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.187262 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1830,"thread_start_us":1663,"threads_started":1}
I20260812 02:43:13.187914 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.181859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.17.196.195:36911)
0812 02:43:13.183519 (+ 1660us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.183530 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.184081 (+ 551us) server_negotiation.cc:407] Connection header received
0812 02:43:13.184169 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.184180 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.184417 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.184590 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.186449 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.186458 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.186469 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.186488 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.186553 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.186578 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.186764 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.187261 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.187351 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1454,"thread_start_us":1262,"threads_started":1}
I20260812 02:43:13.188715 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.181089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:36911 (local address 127.0.0.1:39532)
0812 02:43:13.183807 (+ 2718us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:13.183825 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:13.183934 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:13.185263 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:13.185270 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:13.185296 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:13.186169 (+ 873us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:43:13.186173 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:13.186192 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:43:13.186244 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:13.186826 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:13.186830 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:13.186834 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 02:43:13.187949 (+ 1115us) client_negotiation.cc:240] Negotiation successful
0812 02:43:13.188084 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2542,"thread_start_us":993,"threads_started":1}
I20260812 02:43:13.189642 23867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.180673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:36821 (local address 127.0.0.1:58514)
0812 02:43:13.182100 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:13.182177 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:13.182374 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:13.183795 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:13.183804 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:13.183913 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:13.184899 (+ 986us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:43:13.184903 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:13.184937 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:43:13.185060 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:13.186803 (+ 1743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:13.186817 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:13.186825 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 02:43:13.189026 (+ 2201us) client_negotiation.cc:240] Negotiation successful
0812 02:43:13.189099 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"thread_start_us":2782,"threads_started":1}
I20260812 02:43:13.194607 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.183892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:42697 (local address 127.0.0.1:46678)
0812 02:43:13.189327 (+ 5435us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:13.189347 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:13.189520 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:13.191200 (+ 1680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:13.191205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:13.191231 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:13.193193 (+ 1962us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:43:13.193197 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:13.193215 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:43:13.193269 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:13.193893 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:13.193899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:13.193904 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 02:43:13.194069 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 02:43:13.194132 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5292,"thread_start_us":5134,"threads_started":1}
I20260812 02:43:13.194015 23774 tablet_service.cc:1467] Processing CreateTablet for tablet 2a064ce3801b49f690a6cd3bc35a1788 (DEFAULT_TABLE table=test-table [id=0a136fade9364fa9b9c74bdce0166a3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:13.194267 23622 tablet_service.cc:1467] Processing CreateTablet for tablet 2a064ce3801b49f690a6cd3bc35a1788 (DEFAULT_TABLE table=test-table [id=0a136fade9364fa9b9c74bdce0166a3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:13.195416 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.184623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.17.196.193:42697)
0812 02:43:13.188927 (+ 4304us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.188936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.189666 (+ 730us) server_negotiation.cc:407] Connection header received
0812 02:43:13.190513 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.190527 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.190778 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.190988 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.193469 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.193506 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.193525 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.193550 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.193617 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.193642 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.193828 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.194878 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.195017 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3985,"thread_start_us":3834,"threads_started":1}
I20260812 02:43:13.195614 23622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a064ce3801b49f690a6cd3bc35a1788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:13.195724 23774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a064ce3801b49f690a6cd3bc35a1788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:13.199887 23475 tablet_service.cc:1467] Processing CreateTablet for tablet 2a064ce3801b49f690a6cd3bc35a1788 (DEFAULT_TABLE table=test-table [id=0a136fade9364fa9b9c74bdce0166a3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:13.201681 23475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a064ce3801b49f690a6cd3bc35a1788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:13.208935 23872 tablet_bootstrap.cc:492] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Bootstrap starting.
I20260812 02:43:13.212697 23873 tablet_bootstrap.cc:492] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Bootstrap starting.
I20260812 02:43:13.215361 23872 tablet_bootstrap.cc:654] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:13.216897 23873 tablet_bootstrap.cc:654] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:13.216882 23872 log.cc:824] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:13.218554 23873 log.cc:824] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:13.219692 23875 tablet_bootstrap.cc:492] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Bootstrap starting.
I20260812 02:43:13.220839 23872 tablet_bootstrap.cc:492] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: No bootstrap required, opened a new log
I20260812 02:43:13.221172 23872 ts_tablet_manager.cc:1397] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260812 02:43:13.222079 23873 tablet_bootstrap.cc:492] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: No bootstrap required, opened a new log
I20260812 02:43:13.222412 23873 ts_tablet_manager.cc:1397] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260812 02:43:13.223824 23875 tablet_bootstrap.cc:654] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:13.225258 23875 log.cc:824] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:13.229758 23875 tablet_bootstrap.cc:492] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: No bootstrap required, opened a new log
I20260812 02:43:13.230080 23875 ts_tablet_manager.cc:1397] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260812 02:43:13.231313 23873 raft_consensus.cc:348] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.231770 23873 raft_consensus.cc:374] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:13.231913 23873 raft_consensus.cc:729] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ae2eb7695542ca80622352a5b0ca38, State: Initialized, Role: FOLLOWER
I20260812 02:43:13.232648 23873 consensus_queue.cc:260] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.233012 23872 raft_consensus.cc:348] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.233520 23872 raft_consensus.cc:374] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:13.233636 23872 raft_consensus.cc:729] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ca86b7d9a24c539dda104282b604c9, State: Initialized, Role: FOLLOWER
I20260812 02:43:13.234162 23849 heartbeater.cc:503] Master 127.17.196.254:39305 was elected leader, sending a full tablet report...
I20260812 02:43:13.234755 23873 ts_tablet_manager.cc:1428] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260812 02:43:13.235369 23872 consensus_queue.cc:260] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.236997 23693 heartbeater.cc:503] Master 127.17.196.254:39305 was elected leader, sending a full tablet report...
I20260812 02:43:13.237855 23872 ts_tablet_manager.cc:1428] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260812 02:43:13.242323 23875 raft_consensus.cc:348] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.242784 23875 raft_consensus.cc:374] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:13.242914 23875 raft_consensus.cc:729] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91acc033b2c4cfdb9963f01715e110e, State: Initialized, Role: FOLLOWER
I20260812 02:43:13.243626 23875 consensus_queue.cc:260] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.245280 23875 ts_tablet_manager.cc:1428] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260812 02:43:13.340498 23694 tablet.cc:2369] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:13.359087 23878 raft_consensus.cc:482] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 02:43:13.359401 23550 tablet.cc:2369] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:13.359516 23878 raft_consensus.cc:504] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.361754 23878 leader_election.cc:288] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7ca86b7d9a24c539dda104282b604c9 (127.17.196.194:36821), f91acc033b2c4cfdb9963f01715e110e (127.17.196.193:42697)
I20260812 02:43:13.365149 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.362300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:36315 (local address 127.17.196.194:36821)
0812 02:43:13.362545 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.362552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.362736 (+ 184us) server_negotiation.cc:407] Connection header received
0812 02:43:13.362994 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.363001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.363072 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.363193 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.364466 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.364471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.364474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.364490 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.364528 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.364544 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.364710 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.364843 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.364916 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:43:13.365193 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.362278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:36821 (local address 127.17.196.195:36315)
0812 02:43:13.362625 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:13.362643 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:13.362797 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:13.363410 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:13.363416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:13.363450 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:13.364195 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:43:13.364198 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:13.364220 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:43:13.364291 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:13.364696 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:13.364700 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:13.364704 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 02:43:13.364843 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 02:43:13.364915 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 02:43:13.366207 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.363051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:38941 (local address 127.17.196.193:42697)
0812 02:43:13.363263 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.363273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.363778 (+ 505us) server_negotiation.cc:407] Connection header received
0812 02:43:13.363899 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.363906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.363980 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.364085 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.365404 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.365411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.365416 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.365440 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.365516 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.365537 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.365638 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.365846 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.365917 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 02:43:13.366405 23881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.362896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:42697 (local address 127.17.196.195:38941)
0812 02:43:13.363631 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:13.363656 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:13.363776 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:13.364156 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:13.364162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:13.364187 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:13.364962 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:43:13.364965 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:13.364981 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:43:13.365058 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:13.365716 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:13.365720 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:13.365722 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 02:43:13.365907 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 02:43:13.365962 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":403,"threads_started":1}
I20260812 02:43:13.366207 23643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a064ce3801b49f690a6cd3bc35a1788" candidate_uuid: "28ae2eb7695542ca80622352a5b0ca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ca86b7d9a24c539dda104282b604c9" is_pre_election: true
I20260812 02:43:13.366866 23643 raft_consensus.cc:2454] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28ae2eb7695542ca80622352a5b0ca38 in term 0.
I20260812 02:43:13.367475 23497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a064ce3801b49f690a6cd3bc35a1788" candidate_uuid: "28ae2eb7695542ca80622352a5b0ca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91acc033b2c4cfdb9963f01715e110e" is_pre_election: true
I20260812 02:43:13.367895 23717 leader_election.cc:302] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ae2eb7695542ca80622352a5b0ca38, b7ca86b7d9a24c539dda104282b604c9; no voters:
I20260812 02:43:13.367990 23497 raft_consensus.cc:2454] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28ae2eb7695542ca80622352a5b0ca38 in term 0.
I20260812 02:43:13.368360 23878 raft_consensus.cc:2781] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:13.368597 23878 raft_consensus.cc:482] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:13.368728 23878 raft_consensus.cc:3037] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 0 FOLLOWER]: Advancing to term 1
W20260812 02:43:13.370965 23850 tablet.cc:2369] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:13.371758 23878 raft_consensus.cc:504] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.372538 23878 leader_election.cc:288] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [CANDIDATE]: Term 1 election: Requested vote from peers b7ca86b7d9a24c539dda104282b604c9 (127.17.196.194:36821), f91acc033b2c4cfdb9963f01715e110e (127.17.196.193:42697)
I20260812 02:43:13.373191 23497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a064ce3801b49f690a6cd3bc35a1788" candidate_uuid: "28ae2eb7695542ca80622352a5b0ca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91acc033b2c4cfdb9963f01715e110e"
I20260812 02:43:13.373234 23643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a064ce3801b49f690a6cd3bc35a1788" candidate_uuid: "28ae2eb7695542ca80622352a5b0ca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ca86b7d9a24c539dda104282b604c9"
I20260812 02:43:13.373462 23643 raft_consensus.cc:3037] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:13.373451 23497 raft_consensus.cc:3037] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:13.377460 23497 raft_consensus.cc:2454] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28ae2eb7695542ca80622352a5b0ca38 in term 1.
I20260812 02:43:13.377460 23643 raft_consensus.cc:2454] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28ae2eb7695542ca80622352a5b0ca38 in term 1.
I20260812 02:43:13.378142 23717 leader_election.cc:302] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ae2eb7695542ca80622352a5b0ca38, b7ca86b7d9a24c539dda104282b604c9; no voters:
I20260812 02:43:13.378515 23878 raft_consensus.cc:2781] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:13.379065 23878 raft_consensus.cc:686] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 1 LEADER]: Becoming Leader. State: Replica: 28ae2eb7695542ca80622352a5b0ca38, State: Running, Role: LEADER
I20260812 02:43:13.379609 23878 consensus_queue.cc:237] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } }
I20260812 02:43:13.384994 23341 catalog_manager.cc:5495] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28ae2eb7695542ca80622352a5b0ca38 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "28ae2eb7695542ca80622352a5b0ca38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28ae2eb7695542ca80622352a5b0ca38" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 36911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 } health_report { overall_health: UNKNOWN } } }
I20260812 02:43:13.399842 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.397587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39540 (local address 127.17.196.195:36911)
0812 02:43:13.397777 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.397782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.397965 (+ 183us) server_negotiation.cc:407] Connection header received
0812 02:43:13.398163 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.398167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.398265 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.398353 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.399280 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.399285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.399289 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.399305 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.399343 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.399356 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.399476 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.399567 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.399623 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:43:13.408921 23643 raft_consensus.cc:1261] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 1 FOLLOWER]: Refusing update from remote peer 28ae2eb7695542ca80622352a5b0ca38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:13.408917 23497 raft_consensus.cc:1261] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 1 FOLLOWER]: Refusing update from remote peer 28ae2eb7695542ca80622352a5b0ca38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:13.410079 23883 consensus_queue.cc:1035] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f91acc033b2c4cfdb9963f01715e110e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:13.410523 23878 consensus_queue.cc:1035] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ca86b7d9a24c539dda104282b604c9" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:13.424235 23886 mvcc.cc:205] Tried to move back new op lower bound from 7317514622592094208 to 7317514622487134208. Current Snapshot: MvccSnapshot[applied={T|T < 7317514622592094208}]
I20260812 02:43:13.424734 23887 mvcc.cc:205] Tried to move back new op lower bound from 7317514622592094208 to 7317514622487134208. Current Snapshot: MvccSnapshot[applied={T|T < 7317514622592094208}]
I20260812 02:43:13.438982 23341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:13.439321 23341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:13.443677 23341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0961f26cf80f4756a0cfe99ed91cda77: Sending DeleteTablet for 3 replicas of tablet 2a064ce3801b49f690a6cd3bc35a1788
I20260812 02:43:13.445040 23774 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a064ce3801b49f690a6cd3bc35a1788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:13 UTC) from {username='slave'} at 127.0.0.1:39532
I20260812 02:43:13.445501 23475 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a064ce3801b49f690a6cd3bc35a1788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:13 UTC) from {username='slave'} at 127.0.0.1:46678
I20260812 02:43:13.445580 23622 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a064ce3801b49f690a6cd3bc35a1788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:13 UTC) from {username='slave'} at 127.0.0.1:58514
I20260812 02:43:13.446347 23898 tablet_replica.cc:330] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: stopping tablet replica
I20260812 02:43:13.446504 23899 tablet_replica.cc:330] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: stopping tablet replica
I20260812 02:43:13.447031 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 23406
I20260812 02:43:13.447074 23899 raft_consensus.cc:2229] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:13.447288 23898 raft_consensus.cc:2229] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:43:13.447607 23899 raft_consensus.cc:2258] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:13.448321 23898 raft_consensus.cc:2258] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:13.450197 23899 ts_tablet_manager.cc:1905] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:13.458796 23898 ts_tablet_manager.cc:1905] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:13.460427 23899 ts_tablet_manager.cc:1918] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:13.460630 23899 log.cc:1196] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/2a064ce3801b49f690a6cd3bc35a1788
I20260812 02:43:13.461259 23899 ts_tablet_manager.cc:1939] T 2a064ce3801b49f690a6cd3bc35a1788 P f91acc033b2c4cfdb9963f01715e110e: Deleting consensus metadata
I20260812 02:43:13.462627 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.448889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.17.196.193:42697)
0812 02:43:13.449288 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.449294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.459654 (+ 10360us) server_negotiation.cc:407] Connection header received
0812 02:43:13.459990 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.459997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.460123 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.460208 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.461775 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.461780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.461785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.461803 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.461845 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.461864 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.461977 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.462262 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.462323 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260812 02:43:13.463445 23327 catalog_manager.cc:4841] TS f91acc033b2c4cfdb9963f01715e110e (127.17.196.193:42697): tablet 2a064ce3801b49f690a6cd3bc35a1788 (table test-table [id=0a136fade9364fa9b9c74bdce0166a3b]) successfully deleted
I20260812 02:43:13.463732 23542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:13.464359 23900 tablet_replica.cc:330] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: stopping tablet replica
I20260812 02:43:13.464944 23900 raft_consensus.cc:2229] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:13.465751 23900 raft_consensus.cc:2258] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:13.469969 23898 ts_tablet_manager.cc:1918] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:13.470207 23898 log.cc:1196] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/2a064ce3801b49f690a6cd3bc35a1788
I20260812 02:43:13.471032 23898 ts_tablet_manager.cc:1939] T 2a064ce3801b49f690a6cd3bc35a1788 P 28ae2eb7695542ca80622352a5b0ca38: Deleting consensus metadata
I20260812 02:43:13.471616 23900 ts_tablet_manager.cc:1905] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:13.473203 23324 catalog_manager.cc:4841] TS 28ae2eb7695542ca80622352a5b0ca38 (127.17.196.195:36911): tablet 2a064ce3801b49f690a6cd3bc35a1788 (table test-table [id=0a136fade9364fa9b9c74bdce0166a3b]) successfully deleted
I20260812 02:43:13.479624 23900 ts_tablet_manager.cc:1918] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:13.479826 23900 log.cc:1196] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/2a064ce3801b49f690a6cd3bc35a1788
I20260812 02:43:13.480450 23900 ts_tablet_manager.cc:1939] T 2a064ce3801b49f690a6cd3bc35a1788 P b7ca86b7d9a24c539dda104282b604c9: Deleting consensus metadata
I20260812 02:43:13.482774 23327 catalog_manager.cc:4841] TS b7ca86b7d9a24c539dda104282b604c9 (127.17.196.194:36821): tablet 2a064ce3801b49f690a6cd3bc35a1788 (table test-table [id=0a136fade9364fa9b9c74bdce0166a3b]) successfully deleted
I20260812 02:43:13.577344 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 23406
I20260812 02:43:13.612479 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 23553
I20260812 02:43:13.615809 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.613402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58526 (local address 127.17.196.194:36821)
0812 02:43:13.613618 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.613624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.613645 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:43:13.613867 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.613873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.613968 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.614092 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.615102 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.615107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.615110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.615127 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.615162 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.615174 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.615316 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.615448 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.615519 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 02:43:13.616492 23687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:13.730780 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 23553
I20260812 02:43:13.755434 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 23697
I20260812 02:43:13.759101 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:13.756409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39546 (local address 127.17.196.195:36911)
0812 02:43:13.756646 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:13.756653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:13.756680 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:43:13.756897 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:13.756903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:13.756973 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:13.757102 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:13.758250 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:13.758256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:13.758261 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:13.758282 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:13.758332 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:13.758352 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:13.758540 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:13.758643 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 02:43:13.758720 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 02:43:13.760210 23844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:13.883028 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 23697
I20260812 02:43:13.909404 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 23309
I20260812 02:43:13.911029 23374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:14.028311 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 23309
2026-08-12T02:43:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3041 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 02:43:14.079568 18195 mini_oidc.cc:152] Starting JWKS server
I20260812 02:43:14.083072 18195 webserver.cc:460] Webserver started at https://127.0.0.1:34195/ using document root <none> and password file <none>
I20260812 02:43:14.083523 18195 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 02:43:14.084345 18195 webserver.cc:460] Webserver started at http://127.0.0.1:46237/ using document root <none> and password file <none>
2026-08-12T02:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:14Z Disabled control of system clock
I20260812 02:43:14.123870 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:41053
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:42091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:41053
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34195/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/testchainca.cert with env {}
W20260812 02:43:14.476908 23915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:14.477326 23915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:14.477536 23915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:14.485121 23915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:14.485292 23915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:14.485378 23915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:14.485527 23915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:14.485600 23915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 02:43:14.485695 23915 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 02:43:14.485773 23915 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34195/jwks.json
I20260812 02:43:14.496112 23915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:41053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:41053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34195/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:14.497599 23915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:14.499070 23915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:14.512287 23921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:14.512188 23924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:14.512162 23922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:14.513157 23915 server_base.cc:1034] running on GCE node
I20260812 02:43:14.514186 23915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:14.515779 23915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:14.517030 23915 hybrid_clock.cc:648] HybridClock initialized: now 1786502594516978 us; error 47 us; skew 500 ppm
I20260812 02:43:14.517442 23915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:14.523681 23915 webserver.cc:460] Webserver started at http://127.17.196.254:33353/ using document root <none> and password file <none>
I20260812 02:43:14.524466 23915 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:14.524622 23915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:14.524961 23915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:14.528101 23915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "732c53c51e5b49a8b244acb1d768e402"
format_stamp: "Formatted at 2026-08-12 02:43:14 on dist-test-slave-ss4n"
server_key: "0dfab64233a5fcb4119b4c9ff1d032b4"
server_key_iv: "451bc72521a5425ae157c66aaa947be8"
server_key_version: "encryptionkey@0"
I20260812 02:43:14.528932 23915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "732c53c51e5b49a8b244acb1d768e402"
format_stamp: "Formatted at 2026-08-12 02:43:14 on dist-test-slave-ss4n"
server_key: "0dfab64233a5fcb4119b4c9ff1d032b4"
server_key_iv: "451bc72521a5425ae157c66aaa947be8"
server_key_version: "encryptionkey@0"
I20260812 02:43:14.534502 23915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:43:14.538354 23932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:14.540254 23915 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:43:14.540513 23915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "732c53c51e5b49a8b244acb1d768e402"
format_stamp: "Formatted at 2026-08-12 02:43:14 on dist-test-slave-ss4n"
server_key: "0dfab64233a5fcb4119b4c9ff1d032b4"
server_key_iv: "451bc72521a5425ae157c66aaa947be8"
server_key_version: "encryptionkey@0"
I20260812 02:43:14.540839 23915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:14.580389 23915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:14.581411 23915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:14.581835 23915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:14.603389 23915 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:41053
I20260812 02:43:14.603401 23988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:41053 every 8 connection(s)
I20260812 02:43:14.605006 23915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:14.609658 23989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:14.611600 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 23915
I20260812 02:43:14.611867 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:14.612751 18195 external_mini_cluster.cc:1428] Setting key 27d09c68198fd69e3bb166b5dbfa189e
I20260812 02:43:14.626950 23989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402: Bootstrap starting.
I20260812 02:43:14.629460 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:14.615183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.17.196.254:41053)
0812 02:43:14.616373 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:14.616384 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:14.616447 (+ 63us) server_negotiation.cc:407] Connection header received
0812 02:43:14.618291 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:14.618318 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:14.618870 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:14.619235 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:14.620161 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:14.621786 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:14.623948 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:14.624798 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:14.624962 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:14.626573 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:14.626583 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:14.626600 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:14.627289 (+ 689us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:14.627386 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:14.627410 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:14.627587 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:14.627831 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 02:43:14.628223 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":457,"threads_started":1}
I20260812 02:43:14.630954 23989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:14.632150 23989 log.cc:824] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:14.633906 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:43:14.635380 23989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402: No bootstrap required, opened a new log
I20260812 02:43:14.644001 23989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c53c51e5b49a8b244acb1d768e402" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 41053 } }
I20260812 02:43:14.644456 23989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:14.644575 23989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732c53c51e5b49a8b244acb1d768e402, State: Initialized, Role: FOLLOWER
I20260812 02:43:14.645179 23989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c53c51e5b49a8b244acb1d768e402" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 41053 } }
I20260812 02:43:14.645433 23989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:14.645675 23989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:14.645891 23989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:14.648536 23989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c53c51e5b49a8b244acb1d768e402" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 41053 } }
I20260812 02:43:14.649209 23989 leader_election.cc:302] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 732c53c51e5b49a8b244acb1d768e402; no voters:
I20260812 02:43:14.649927 23989 leader_election.cc:288] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:14.650089 23994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:14.650924 23994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [term 1 LEADER]: Becoming Leader. State: Replica: 732c53c51e5b49a8b244acb1d768e402, State: Running, Role: LEADER
I20260812 02:43:14.651515 23994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c53c51e5b49a8b244acb1d768e402" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 41053 } }
I20260812 02:43:14.652134 23989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:14.654992 23996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 732c53c51e5b49a8b244acb1d768e402. Latest consensus state: current_term: 1 leader_uuid: "732c53c51e5b49a8b244acb1d768e402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c53c51e5b49a8b244acb1d768e402" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 41053 } } }
I20260812 02:43:14.655359 23995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "732c53c51e5b49a8b244acb1d768e402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c53c51e5b49a8b244acb1d768e402" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 41053 } } }
I20260812 02:43:14.661904 23995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:14.661896 23996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:14.663520 24004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:14.668605 24004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:14.678318 24004 catalog_manager.cc:1347] Generated new cluster ID: 650cde44367d4fe3a1b18a3d63f8567b
I20260812 02:43:14.678450 24004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:14.705041 24004 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:14.706919 24004 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:14.716852 24004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 732c53c51e5b49a8b244acb1d768e402: Generated new TSK 0
I20260812 02:43:14.717976 24004 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 02:43:14.761824 24019 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 02:43:14.762073 24019 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 02:43:14.765040 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:14.736886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.17.196.254:41053)
0812 02:43:14.737088 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:14.737094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:14.737521 (+ 427us) server_negotiation.cc:407] Connection header received
0812 02:43:14.737954 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:14.737962 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:14.738097 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:14.738298 (+ 201us) server_negotiation.cc:226] Negotiated authn=JWT
0812 02:43:14.738943 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:14.740271 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:14.741967 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:14.742796 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:14.742957 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:14.743546 (+ 589us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 02:43:14.764171 (+ 20625us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 02:43:14.764645 (+ 474us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54002: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 02:43:14.765194 23992 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54002: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 02:43:14.766607 24018 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:14.736682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:41053 (local address 127.0.0.1:54002)
0812 02:43:14.737378 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:14.737396 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:14.737628 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:14.738332 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:14.738336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:14.738346 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 02:43:14.738772 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:14.738781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:14.740480 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:14.740485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:14.741795 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:14.741811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:14.742990 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:14.742995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:14.743138 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:14.743209 (+ 71us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 02:43:14.764585 (+ 21376us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 02:43:14.764914 (+ 329us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.196.254:41053: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":348,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.196.254:41053: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 02:43:14.768448 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 23915
I20260812 02:43:14.769796 23983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:14.868752 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 23915
2026-08-12T02:43:14Z chronyd exiting
I20260812 02:43:14.921152 18195 test_util.cc:175] -----------------------------------------------
I20260812 02:43:14.921340 18195 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502547576936-18195-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (847 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 02:43:14.925985 18195 mini_oidc.cc:152] Starting JWKS server
I20260812 02:43:14.928164 18195 webserver.cc:460] Webserver started at https://127.0.0.1:38511/ using document root <none> and password file <none>
I20260812 02:43:14.928462 18195 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 02:43:14.929281 18195 webserver.cc:460] Webserver started at http://127.0.0.1:46721/ using document root <none> and password file <none>
2026-08-12T02:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:14Z Disabled control of system clock
I20260812 02:43:14.969791 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:35231
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:35231
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38511/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/testchainca.cert with env {}
W20260812 02:43:15.342778 24026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:15.343191 24026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:15.343349 24026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:15.352573 24026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:15.352774 24026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:15.352872 24026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:15.352967 24026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:15.353061 24026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 02:43:15.353152 24026 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 02:43:15.353232 24026 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38511/jwks.json
I20260812 02:43:15.364286 24026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:35231
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:35231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38511/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:15.365947 24026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:15.367472 24026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:15.381319 24034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:15.381428 24032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:15.381347 24031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:15.387791 24026 server_base.cc:1034] running on GCE node
I20260812 02:43:15.388806 24026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:15.390614 24026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:15.391903 24026 hybrid_clock.cc:648] HybridClock initialized: now 1786502595391836 us; error 58 us; skew 500 ppm
I20260812 02:43:15.392371 24026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:15.395164 24026 webserver.cc:460] Webserver started at http://127.17.196.254:41653/ using document root <none> and password file <none>
I20260812 02:43:15.395987 24026 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:15.396137 24026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:15.396505 24026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:15.399793 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "cb86508e5c794ea0ac78ce00708cb7a9"
format_stamp: "Formatted at 2026-08-12 02:43:15 on dist-test-slave-ss4n"
server_key: "182147b3095a16f40afac7dfac972aef"
server_key_iv: "3882ae38e300bffb032148b3c93779a4"
server_key_version: "encryptionkey@0"
I20260812 02:43:15.400688 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "cb86508e5c794ea0ac78ce00708cb7a9"
format_stamp: "Formatted at 2026-08-12 02:43:15 on dist-test-slave-ss4n"
server_key: "182147b3095a16f40afac7dfac972aef"
server_key_iv: "3882ae38e300bffb032148b3c93779a4"
server_key_version: "encryptionkey@0"
I20260812 02:43:15.406972 24026 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 02:43:15.411141 24042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:15.412962 24026 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:43:15.413228 24026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "cb86508e5c794ea0ac78ce00708cb7a9"
format_stamp: "Formatted at 2026-08-12 02:43:15 on dist-test-slave-ss4n"
server_key: "182147b3095a16f40afac7dfac972aef"
server_key_iv: "3882ae38e300bffb032148b3c93779a4"
server_key_version: "encryptionkey@0"
I20260812 02:43:15.413509 24026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:15.454939 24026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:15.455933 24026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:15.456308 24026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:15.479310 24026 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:35231
I20260812 02:43:15.479324 24101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:35231 every 8 connection(s)
I20260812 02:43:15.480958 24026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:15.486222 24102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:15.487862 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 24026
I20260812 02:43:15.488116 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:15.489053 18195 external_mini_cluster.cc:1428] Setting key 320b6d9923703cde20d0edf586bd00c5
I20260812 02:43:15.503136 24102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9: Bootstrap starting.
I20260812 02:43:15.505640 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:15.491529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.17.196.254:35231)
0812 02:43:15.492830 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:15.492846 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:15.492898 (+ 52us) server_negotiation.cc:407] Connection header received
0812 02:43:15.494727 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:15.494764 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:15.495508 (+ 744us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:15.496006 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:15.496544 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:15.498508 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:15.499984 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:15.501084 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:15.501259 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:15.502577 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:15.502589 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:15.502608 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:15.503365 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:15.503501 (+ 136us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:15.503531 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:15.503652 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:15.503878 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 02:43:15.504231 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":548,"threads_started":1}
I20260812 02:43:15.507217 24102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:15.508630 24102 log.cc:824] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:15.510218 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:43:15.511859 24102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9: No bootstrap required, opened a new log
I20260812 02:43:15.520592 24102 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86508e5c794ea0ac78ce00708cb7a9" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35231 } }
I20260812 02:43:15.520991 24102 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:15.521128 24102 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb86508e5c794ea0ac78ce00708cb7a9, State: Initialized, Role: FOLLOWER
I20260812 02:43:15.521843 24102 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86508e5c794ea0ac78ce00708cb7a9" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35231 } }
I20260812 02:43:15.522099 24102 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:15.522275 24102 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:15.522464 24102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:15.525100 24102 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86508e5c794ea0ac78ce00708cb7a9" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35231 } }
I20260812 02:43:15.525755 24102 leader_election.cc:302] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb86508e5c794ea0ac78ce00708cb7a9; no voters:
I20260812 02:43:15.526386 24102 leader_election.cc:288] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:15.526607 24107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:15.527539 24107 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [term 1 LEADER]: Becoming Leader. State: Replica: cb86508e5c794ea0ac78ce00708cb7a9, State: Running, Role: LEADER
I20260812 02:43:15.528146 24107 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86508e5c794ea0ac78ce00708cb7a9" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35231 } }
I20260812 02:43:15.528668 24102 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:15.531342 24110 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb86508e5c794ea0ac78ce00708cb7a9. Latest consensus state: current_term: 1 leader_uuid: "cb86508e5c794ea0ac78ce00708cb7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86508e5c794ea0ac78ce00708cb7a9" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35231 } } }
I20260812 02:43:15.531795 24110 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:15.532455 24109 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb86508e5c794ea0ac78ce00708cb7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86508e5c794ea0ac78ce00708cb7a9" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35231 } } }
I20260812 02:43:15.532799 24109 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:15.536537 24115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:15.542845 24115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:15.553629 24115 catalog_manager.cc:1347] Generated new cluster ID: a64c4dcc3a904482a2815b5e987c1877
I20260812 02:43:15.553766 24115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:15.568585 24115 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:15.570612 24115 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:15.582518 24115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb86508e5c794ea0ac78ce00708cb7a9: Generated new TSK 0
I20260812 02:43:15.583702 24115 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 02:43:15.628113 24135 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 02:43:15.628276 24135 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 02:43:15.629714 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:15.608142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.17.196.254:35231)
0812 02:43:15.608397 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:15.608414 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:15.608820 (+ 406us) server_negotiation.cc:407] Connection header received
0812 02:43:15.609178 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:15.609183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:15.609272 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:15.609428 (+ 156us) server_negotiation.cc:226] Negotiated authn=JWT
0812 02:43:15.610083 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:15.610907 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:15.612965 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:15.613435 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:15.613632 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:15.614285 (+ 653us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 02:43:15.628924 (+ 14639us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 02:43:15.629362 (+ 438us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33284: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 02:43:15.629863 24105 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33284: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 02:43:15.630496 24134 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:15.607962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:35231 (local address 127.0.0.1:33284)
0812 02:43:15.608684 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:15.608708 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:15.608888 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:15.609505 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:15.609515 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:15.609532 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0812 02:43:15.609857 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:15.609872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:15.611068 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:15.611074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:15.612784 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:15.612800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:15.613710 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:15.613717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:15.613942 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:15.613998 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 02:43:15.629244 (+ 15246us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 02:43:15.629610 (+ 366us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.196.254:35231: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":367,"threads_started":1}
I20260812 02:43:15.631732 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 24026
I20260812 02:43:15.633244 24097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:15.732925 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 24026
2026-08-12T02:43:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (870 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 02:43:15.796088 18195 mini_oidc.cc:152] Starting JWKS server
I20260812 02:43:15.798115 18195 webserver.cc:460] Webserver started at https://127.0.0.1:39253/ using document root <none> and password file <none>
I20260812 02:43:15.798341 18195 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 02:43:15.798959 18195 webserver.cc:460] Webserver started at http://127.0.0.1:34061/ using document root <none> and password file <none>
2026-08-12T02:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:15Z Disabled control of system clock
I20260812 02:43:15.837630 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42777
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:42777
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39253/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 02:43:16.195361 24142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:16.195725 24142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:16.195909 24142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:16.204082 24142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:16.204212 24142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:16.204293 24142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:16.204406 24142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:16.204473 24142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 02:43:16.204555 24142 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 02:43:16.204614 24142 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39253/jwks.json
I20260812 02:43:16.214465 24142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:42777
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39253/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:16.215896 24142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:16.217959 24142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:16.230945 24148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:16.231045 24149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:16.231009 24151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:16.232251 24142 server_base.cc:1034] running on GCE node
I20260812 02:43:16.233202 24142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:16.235000 24142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:16.236254 24142 hybrid_clock.cc:648] HybridClock initialized: now 1786502596236212 us; error 46 us; skew 500 ppm
I20260812 02:43:16.236711 24142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:16.239750 24142 webserver.cc:460] Webserver started at http://127.17.196.254:37961/ using document root <none> and password file <none>
I20260812 02:43:16.240625 24142 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:16.240741 24142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:16.241206 24142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:16.245047 24142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "edf1d93088f84912a1f6d363865edd57"
format_stamp: "Formatted at 2026-08-12 02:43:16 on dist-test-slave-ss4n"
server_key: "4c47fc33f528497c7845642a9fd5cb74"
server_key_iv: "e700dc070a4303c57afe88dfacd79d27"
server_key_version: "encryptionkey@0"
I20260812 02:43:16.246137 24142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "edf1d93088f84912a1f6d363865edd57"
format_stamp: "Formatted at 2026-08-12 02:43:16 on dist-test-slave-ss4n"
server_key: "4c47fc33f528497c7845642a9fd5cb74"
server_key_iv: "e700dc070a4303c57afe88dfacd79d27"
server_key_version: "encryptionkey@0"
I20260812 02:43:16.252126 24142 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260812 02:43:16.256228 24158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:16.258030 24142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:43:16.258255 24142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "edf1d93088f84912a1f6d363865edd57"
format_stamp: "Formatted at 2026-08-12 02:43:16 on dist-test-slave-ss4n"
server_key: "4c47fc33f528497c7845642a9fd5cb74"
server_key_iv: "e700dc070a4303c57afe88dfacd79d27"
server_key_version: "encryptionkey@0"
I20260812 02:43:16.258520 24142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:16.273685 24142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:16.274629 24142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:16.274984 24142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:16.296002 24142 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:42777
I20260812 02:43:16.296028 24220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:42777 every 8 connection(s)
I20260812 02:43:16.297473 24142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:16.301458 24221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:16.304526 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 24142
I20260812 02:43:16.304760 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:16.305622 18195 external_mini_cluster.cc:1428] Setting key 666dd619df026356526f4e00b5ffe15e
I20260812 02:43:16.315791 24221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57: Bootstrap starting.
I20260812 02:43:16.320526 24221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:16.320976 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:16.308121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35360 (local address 127.17.196.254:42777)
0812 02:43:16.309286 (+ 1165us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:16.309299 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:16.309342 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:43:16.311167 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:16.311193 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:16.311693 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:16.312145 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:16.312779 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:16.314218 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:16.315630 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:16.316431 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:16.316692 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:16.318334 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:16.318342 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:16.318357 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:16.318950 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:16.319047 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:16.319070 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:16.319169 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:16.319467 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 02:43:16.319831 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":445,"threads_started":1}
I20260812 02:43:16.322113 24221 log.cc:824] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:16.326325 24221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57: No bootstrap required, opened a new log
I20260812 02:43:16.326480 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:43:16.336547 24221 raft_consensus.cc:348] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf1d93088f84912a1f6d363865edd57" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42777 } }
I20260812 02:43:16.337033 24221 raft_consensus.cc:374] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:16.337163 24221 raft_consensus.cc:729] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf1d93088f84912a1f6d363865edd57, State: Initialized, Role: FOLLOWER
I20260812 02:43:16.337903 24221 consensus_queue.cc:260] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf1d93088f84912a1f6d363865edd57" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42777 } }
I20260812 02:43:16.338181 24221 raft_consensus.cc:388] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:16.338333 24221 raft_consensus.cc:482] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:16.338508 24221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:16.340941 24221 raft_consensus.cc:504] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf1d93088f84912a1f6d363865edd57" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42777 } }
I20260812 02:43:16.341687 24221 leader_election.cc:302] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edf1d93088f84912a1f6d363865edd57; no voters:
I20260812 02:43:16.342497 24221 leader_election.cc:288] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:16.342689 24226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:16.343559 24226 raft_consensus.cc:686] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [term 1 LEADER]: Becoming Leader. State: Replica: edf1d93088f84912a1f6d363865edd57, State: Running, Role: LEADER
I20260812 02:43:16.344189 24226 consensus_queue.cc:237] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf1d93088f84912a1f6d363865edd57" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42777 } }
I20260812 02:43:16.344904 24221 sys_catalog.cc:564] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:16.347504 24228 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edf1d93088f84912a1f6d363865edd57. Latest consensus state: current_term: 1 leader_uuid: "edf1d93088f84912a1f6d363865edd57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf1d93088f84912a1f6d363865edd57" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42777 } } }
I20260812 02:43:16.347999 24228 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:16.348472 24227 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edf1d93088f84912a1f6d363865edd57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf1d93088f84912a1f6d363865edd57" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42777 } } }
I20260812 02:43:16.348830 24227 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:16.352486 24232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:16.359232 24232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:16.370043 24232 catalog_manager.cc:1347] Generated new cluster ID: 7a9cbe2f299f405f9b24ea009d230389
I20260812 02:43:16.370182 24232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:16.396459 24232 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:16.398377 24232 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:16.407586 24232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edf1d93088f84912a1f6d363865edd57: Generated new TSK 0
I20260812 02:43:16.408599 24232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:43:16.432075 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:16.423342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.17.196.254:42777)
0812 02:43:16.423593 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:16.423601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:16.424142 (+ 541us) server_negotiation.cc:407] Connection header received
0812 02:43:16.424452 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:16.424457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:16.424535 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:16.424728 (+ 193us) server_negotiation.cc:226] Negotiated authn=JWT
0812 02:43:16.425232 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:16.426076 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:16.427507 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:16.427981 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:16.428146 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:16.428735 (+ 589us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 02:43:16.431329 (+ 2594us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 02:43:16.431754 (+ 425us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35366: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 02:43:16.432205 24224 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35366: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 02:43:16.434254 24252 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:16.423209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42777 (local address 127.0.0.1:35366)
0812 02:43:16.424006 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:16.424031 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:16.424230 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:16.424734 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:16.424740 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:16.424751 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0812 02:43:16.425059 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:16.425068 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:16.426237 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:16.426243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:16.427329 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:16.427340 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:16.428221 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:16.428230 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:16.428383 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:16.428421 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 02:43:16.431653 (+ 3232us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 02:43:16.433746 (+ 2093us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.196.254:42777: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":1727,"thread_start_us":485,"threads_started":1}
I20260812 02:43:16.435931 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 24142
I20260812 02:43:16.437244 24215 generic_service.cc:196] Checking for leaks (request via RPC)
E20260812 02:43:16.444476 24253 webserver.cc:533] Webserver: sslize failed:
E20260812 02:43:16.444593 24253 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 02:43:16.532869 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 24142
2026-08-12T02:43:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (799 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T02:43:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:16Z Disabled control of system clock
I20260812 02:43:16.632927 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:43509
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:43509
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/testchainca.cert with env {}
W20260812 02:43:16.994561 24259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:16.994912 24259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:16.995075 24259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:17.002882 24259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:17.003005 24259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:17.003091 24259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:17.003188 24259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:17.003285 24259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 02:43:17.003404 24259 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 02:43:17.003472 24259 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 02:43:17.014222 24259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:43509
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:43509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:17.015663 24259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:17.017076 24259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:17.029949 24264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:17.030053 24265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:17.030020 24267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:17.031116 24259 server_base.cc:1034] running on GCE node
I20260812 02:43:17.032023 24259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:17.033810 24259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:17.035112 24259 hybrid_clock.cc:648] HybridClock initialized: now 1786502597035072 us; error 56 us; skew 500 ppm
I20260812 02:43:17.035523 24259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:43:17.038517 24259 webserver.cc:460] Webserver started at http://127.17.196.254:36573/ using document root <none> and password file <none>
I20260812 02:43:17.039467 24259 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:17.039640 24259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:17.040043 24259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:17.043437 24259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "a007f13b025741608a1f79c79611f205"
format_stamp: "Formatted at 2026-08-12 02:43:17 on dist-test-slave-ss4n"
server_key: "690c8855dd55295f815d9cb51f55e752"
server_key_iv: "85d46a60a05c9adc6fbd90aa550471c5"
server_key_version: "encryptionkey@0"
I20260812 02:43:17.044409 24259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "a007f13b025741608a1f79c79611f205"
format_stamp: "Formatted at 2026-08-12 02:43:17 on dist-test-slave-ss4n"
server_key: "690c8855dd55295f815d9cb51f55e752"
server_key_iv: "85d46a60a05c9adc6fbd90aa550471c5"
server_key_version: "encryptionkey@0"
I20260812 02:43:17.050470 24259 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 02:43:17.054522 24274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:17.056273 24259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:43:17.056514 24259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "a007f13b025741608a1f79c79611f205"
format_stamp: "Formatted at 2026-08-12 02:43:17 on dist-test-slave-ss4n"
server_key: "690c8855dd55295f815d9cb51f55e752"
server_key_iv: "85d46a60a05c9adc6fbd90aa550471c5"
server_key_version: "encryptionkey@0"
I20260812 02:43:17.056782 24259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:17.071323 24259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:17.072364 24259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:17.072738 24259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:17.095502 24259 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:43509
I20260812 02:43:17.095517 24330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:43509 every 8 connection(s)
I20260812 02:43:17.097115 24259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:17.101199 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 24259
I20260812 02:43:17.101233 24331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:17.101445 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:17.102227 18195 external_mini_cluster.cc:1428] Setting key 4326a27ff77f0375ab77b69f357fcd78
I20260812 02:43:17.116007 24331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205: Bootstrap starting.
I20260812 02:43:17.118256 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:17.104503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37870 (local address 127.17.196.254:43509)
0812 02:43:17.105799 (+ 1296us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:17.105815 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:17.105868 (+ 53us) server_negotiation.cc:407] Connection header received
0812 02:43:17.107684 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:17.107710 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:17.108250 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:17.108632 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:17.109288 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:17.110816 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:17.112971 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:17.113909 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:17.114083 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:17.115491 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:17.115499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:17.115515 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:43:17.116137 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:17.116246 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:43:17.116270 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:17.116353 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:17.116702 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 02:43:17.117107 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":437,"threads_started":1}
I20260812 02:43:17.119724 24331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:17.120918 24331 log.cc:824] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:17.122674 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:43:17.124298 24331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205: No bootstrap required, opened a new log
I20260812 02:43:17.132550 24331 raft_consensus.cc:348] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a007f13b025741608a1f79c79611f205" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43509 } }
I20260812 02:43:17.132922 24331 raft_consensus.cc:374] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:17.133076 24331 raft_consensus.cc:729] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a007f13b025741608a1f79c79611f205, State: Initialized, Role: FOLLOWER
I20260812 02:43:17.133731 24331 consensus_queue.cc:260] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a007f13b025741608a1f79c79611f205" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43509 } }
I20260812 02:43:17.133973 24331 raft_consensus.cc:388] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:17.134142 24331 raft_consensus.cc:482] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:17.134341 24331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:17.136889 24331 raft_consensus.cc:504] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a007f13b025741608a1f79c79611f205" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43509 } }
I20260812 02:43:17.137395 24331 leader_election.cc:302] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a007f13b025741608a1f79c79611f205; no voters:
I20260812 02:43:17.138053 24331 leader_election.cc:288] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:17.138245 24337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:17.139050 24337 raft_consensus.cc:686] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [term 1 LEADER]: Becoming Leader. State: Replica: a007f13b025741608a1f79c79611f205, State: Running, Role: LEADER
I20260812 02:43:17.139708 24337 consensus_queue.cc:237] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a007f13b025741608a1f79c79611f205" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43509 } }
I20260812 02:43:17.140645 24331 sys_catalog.cc:564] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:17.143036 24338 sys_catalog.cc:455] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a007f13b025741608a1f79c79611f205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a007f13b025741608a1f79c79611f205" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43509 } } }
I20260812 02:43:17.143438 24338 sys_catalog.cc:458] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:17.143154 24339 sys_catalog.cc:455] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a007f13b025741608a1f79c79611f205. Latest consensus state: current_term: 1 leader_uuid: "a007f13b025741608a1f79c79611f205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a007f13b025741608a1f79c79611f205" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43509 } } }
I20260812 02:43:17.143571 24339 sys_catalog.cc:458] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:17.146898 24345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:17.152781 24345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:17.163308 24345 catalog_manager.cc:1347] Generated new cluster ID: 45f15c2be2b24243a80954c8ba8c25a6
I20260812 02:43:17.163446 24345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:17.179005 24345 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:17.180925 24345 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:17.189628 24345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a007f13b025741608a1f79c79611f205: Generated new TSK 0
I20260812 02:43:17.190750 24345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:43:17.647739 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:17.212754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37876 (local address 127.17.196.254:43509)
0812 02:43:17.212979 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:17.212987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:17.213436 (+ 449us) server_negotiation.cc:407] Connection header received
0812 02:43:17.213752 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:17.213761 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:17.213882 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:17.214057 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0812 02:43:17.214542 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:17.215739 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:17.217253 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:17.218011 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:17.218154 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:17.218990 (+ 836us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 02:43:17.646858 (+427868us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 02:43:17.647350 (+ 492us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37876: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 02:43:17.647859 24334 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37876: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 02:43:17.648011 24365 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:17.212589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:43509 (local address 127.0.0.1:37876)
0812 02:43:17.213326 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:17.213343 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:17.213535 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:17.214104 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:17.214107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:17.214118 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0812 02:43:17.214362 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:17.214370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:17.215930 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:17.215935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:17.217066 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:17.217104 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:17.218268 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:17.218273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:17.218550 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:17.218629 (+ 79us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 02:43:17.647233 (+428604us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 02:43:17.647649 (+ 416us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.196.254:43509: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":402,"threads_started":1}
I20260812 02:43:17.649531 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 24259
I20260812 02:43:17.650796 24326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:17.761094 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 24259
2026-08-12T02:43:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1210 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (465 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (465 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:43:18 dist-test-slave-ss4n krb5kdc[24382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:43:18 dist-test-slave-ss4n krb5kdc[24382](info): set up 2 sockets
Aug 12 02:43:18 dist-test-slave-ss4n krb5kdc[24382](info): commencing operation
krb5kdc: starting...
W20260812 02:43:19.934617 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:43:20 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:43:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:43:20.218935 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44459
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:20.583050 24396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:20.583464 24396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:20.583632 24396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:20.591415 24396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:20.591603 24396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:20.591713 24396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:20.591817 24396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:20.591895 24396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:20.602540 24396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:20.604051 24396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:20.605644 24396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:20.619297 24401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:20.619292 24402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:20.619503 24404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:20.620699 24396 server_base.cc:1034] running on GCE node
I20260812 02:43:20.621768 24396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:20.623505 24396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:20.624766 24396 hybrid_clock.cc:648] HybridClock initialized: now 1786502600624708 us; error 47 us; skew 500 ppm
I20260812 02:43:20.625241 24396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:20 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:20.633010 24396 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:43:20.635094 24396 webserver.cc:460] Webserver started at http://127.17.196.254:42905/ using document root <none> and password file <none>
I20260812 02:43:20.635900 24396 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:20.636056 24396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:20.636473 24396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:20.639986 24396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "55e94d52e2f04d00a3f27530fcfcd319"
format_stamp: "Formatted at 2026-08-12 02:43:20 on dist-test-slave-ss4n"
server_key: "5e03aefe532211abacbbbc5f64ca5fa3"
server_key_iv: "04fa8c98b7bd50507e2133c458eceda3"
server_key_version: "encryptionkey@0"
I20260812 02:43:20.640938 24396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "55e94d52e2f04d00a3f27530fcfcd319"
format_stamp: "Formatted at 2026-08-12 02:43:20 on dist-test-slave-ss4n"
server_key: "5e03aefe532211abacbbbc5f64ca5fa3"
server_key_iv: "04fa8c98b7bd50507e2133c458eceda3"
server_key_version: "encryptionkey@0"
I20260812 02:43:20.647388 24396 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 02:43:20.652030 24412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:20.653904 24396 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:43:20.654124 24396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "55e94d52e2f04d00a3f27530fcfcd319"
format_stamp: "Formatted at 2026-08-12 02:43:20 on dist-test-slave-ss4n"
server_key: "5e03aefe532211abacbbbc5f64ca5fa3"
server_key_iv: "04fa8c98b7bd50507e2133c458eceda3"
server_key_version: "encryptionkey@0"
I20260812 02:43:20.654376 24396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:20.679149 24396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:20.683928 24396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:20.684254 24396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:20.707499 24396 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:44459
I20260812 02:43:20.707643 24466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:44459 every 8 connection(s)
I20260812 02:43:20.709146 24396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:20.713469 24467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:20.717712 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 24396
I20260812 02:43:20.717962 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:20.718811 18195 external_mini_cluster.cc:1428] Setting key 742984d479083b81869196754ee07589
I20260812 02:43:20.726365 24467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319: Bootstrap starting.
I20260812 02:43:20.730679 24467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:20.732072 24467 log.cc:824] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:20.736545 24467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319: No bootstrap required, opened a new log
Aug 12 02:43:20 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:20.745349 24467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e94d52e2f04d00a3f27530fcfcd319" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44459 } }
I20260812 02:43:20.745931 24467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:20.746076 24467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e94d52e2f04d00a3f27530fcfcd319, State: Initialized, Role: FOLLOWER
I20260812 02:43:20.746716 24467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e94d52e2f04d00a3f27530fcfcd319" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44459 } }
I20260812 02:43:20.747022 24467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:20.747201 24467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:20.747591 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:20.721295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.17.196.254:44459)
0812 02:43:20.722317 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:20.722331 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:20.722380 (+ 49us) server_negotiation.cc:407] Connection header received
0812 02:43:20.725052 (+ 2672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:20.725085 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:20.725697 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:20.726109 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:20.728161 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:20.730022 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:20.731956 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:20.733079 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:20.733323 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:20.738823 (+ 5500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:20.738834 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:20.738853 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:20.738908 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:20.742470 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:20.743319 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:20.743327 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:20.743338 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:20.743527 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:20.743990 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:20.743995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:20.743998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:20.744643 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:20.744988 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:20.745645 (+ 657us) server_negotiation.cc:299] Negotiation successful
0812 02:43:20.746226 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":512,"threads_started":1}
I20260812 02:43:20.747500 24467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:20.750514 24467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e94d52e2f04d00a3f27530fcfcd319" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44459 } }
I20260812 02:43:20.751111 24467 leader_election.cc:302] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55e94d52e2f04d00a3f27530fcfcd319; no voters:
I20260812 02:43:20.751818 24467 leader_election.cc:288] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:20.751928 24473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:20.752728 24473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [term 1 LEADER]: Becoming Leader. State: Replica: 55e94d52e2f04d00a3f27530fcfcd319, State: Running, Role: LEADER
I20260812 02:43:20.753396 24473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e94d52e2f04d00a3f27530fcfcd319" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44459 } }
I20260812 02:43:20.754376 24467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:20.756362 24474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55e94d52e2f04d00a3f27530fcfcd319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e94d52e2f04d00a3f27530fcfcd319" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44459 } } }
I20260812 02:43:20.758675 24474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:20.757962 24475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55e94d52e2f04d00a3f27530fcfcd319. Latest consensus state: current_term: 1 leader_uuid: "55e94d52e2f04d00a3f27530fcfcd319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e94d52e2f04d00a3f27530fcfcd319" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44459 } } }
I20260812 02:43:20.760383 24481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:20.760382 24475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:20.765900 24481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:20.775491 24481 catalog_manager.cc:1347] Generated new cluster ID: f7057f3fdbed4ac98dae3e4d8efcb408
I20260812 02:43:20.775632 24481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:20.785693 24481 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:20.787997 24481 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:20.798942 24481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55e94d52e2f04d00a3f27530fcfcd319: Generated new TSK 0
I20260812 02:43:20.800045 24481 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:43:20.874212 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44459
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:21.240516 24498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:21.241026 24498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:21.241194 24498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:21.248808 24498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:21.248929 24498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:21.249143 24498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:21.259986 24498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:21.261468 24498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:21.263000 24498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:21.279007 24504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:21.279034 24503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:21.279917 24506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:21.279965 24498 server_base.cc:1034] running on GCE node
I20260812 02:43:21.280787 24498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:21.282167 24498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:21.283429 24498 hybrid_clock.cc:648] HybridClock initialized: now 1786502601283372 us; error 42 us; skew 500 ppm
I20260812 02:43:21.283824 24498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:21 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502601, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:21.287405 24498 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:43:21.288988 24498 webserver.cc:460] Webserver started at http://127.17.196.193:37307/ using document root <none> and password file <none>
I20260812 02:43:21.289752 24498 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:21.289860 24498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:21.290132 24498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:21.293205 24498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "fbe1d7c512174607910eb7554034227a"
format_stamp: "Formatted at 2026-08-12 02:43:21 on dist-test-slave-ss4n"
server_key: "73e5062cbabaab8e7a6cd89e0a5b748a"
server_key_iv: "903a561690e26dfc8ecabda588b18f0c"
server_key_version: "encryptionkey@0"
I20260812 02:43:21.294041 24498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "fbe1d7c512174607910eb7554034227a"
format_stamp: "Formatted at 2026-08-12 02:43:21 on dist-test-slave-ss4n"
server_key: "73e5062cbabaab8e7a6cd89e0a5b748a"
server_key_iv: "903a561690e26dfc8ecabda588b18f0c"
server_key_version: "encryptionkey@0"
I20260812 02:43:21.299890 24498 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:43:21.304131 24514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:21.305972 24498 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:43:21.306190 24498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "fbe1d7c512174607910eb7554034227a"
format_stamp: "Formatted at 2026-08-12 02:43:21 on dist-test-slave-ss4n"
server_key: "73e5062cbabaab8e7a6cd89e0a5b748a"
server_key_iv: "903a561690e26dfc8ecabda588b18f0c"
server_key_version: "encryptionkey@0"
I20260812 02:43:21.306411 24498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:21.323601 24498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:21.328732 24498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:21.329120 24498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:21.330302 24498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:21.331926 24498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:21.332007 24498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:21.332141 24498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:21.332217 24498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:21.371485 24498 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:39093
I20260812 02:43:21.371560 24640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:39093 every 8 connection(s)
I20260812 02:43:21.373282 24498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:21.382731 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 24498
I20260812 02:43:21.382982 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:43:21.383944 18195 external_mini_cluster.cc:1428] Setting key 59cf2c06909081a45046f2b420715ea0
Aug 12 02:43:21 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502601, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:21.397794 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:21.376301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:49385 (local address 127.17.196.254:44459)
0812 02:43:21.376835 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:21.376840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:21.378008 (+ 1168us) server_negotiation.cc:407] Connection header received
0812 02:43:21.379946 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:21.379951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:21.380062 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:21.380174 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:21.382315 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:21.383214 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:21.385783 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:21.386414 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:21.386576 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:21.391380 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:21.391386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:21.391389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:21.391426 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:21.394338 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:21.395233 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:21.395238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:21.395241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:21.395371 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:21.395866 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:21.395876 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:21.395879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:21.396157 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:21.396324 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:21.397142 (+ 818us) server_negotiation.cc:299] Negotiation successful
0812 02:43:21.397390 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":290,"threads_started":1}
I20260812 02:43:21.398850 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:21.376915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44459 (local address 127.17.196.193:49385)
0812 02:43:21.377815 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:21.377857 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:21.379636 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:21.380430 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:21.380439 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:21.381082 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:21.382038 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:21.382065 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:21.383505 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:21.383512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:21.385624 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:21.385641 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:21.386699 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:21.386708 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:21.387044 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:21.387935 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:21.387979 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:21.391087 (+ 3108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:21.394535 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:21.394546 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:21.394568 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:21.395050 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:21.395500 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:21.395505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:21.395509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:21.395730 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:21.396300 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:21.396309 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:21.396708 (+ 399us) client_negotiation.cc:769] Sending connection context
0812 02:43:21.396997 (+ 289us) client_negotiation.cc:240] Negotiation successful
0812 02:43:21.397645 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":346,"threads_started":1}
I20260812 02:43:21.400698 24641 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44459
I20260812 02:43:21.401044 24641 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:21.401885 24641 heartbeater.cc:511] Master 127.17.196.254:44459 requested a full tablet report, sending...
I20260812 02:43:21.404536 24429 ts_manager.cc:194] Registered new tserver with Master: fbe1d7c512174607910eb7554034227a (127.17.196.193:39093)
I20260812 02:43:21.407007 24429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:49385
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:43:21.448844 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44459
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:21.811184 24648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:21.811491 24648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:21.811590 24648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:21.819306 24648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:21.819429 24648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:21.819556 24648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:21.829638 24648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:21.830937 24648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:21.832332 24648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:21.852895 24654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:21.852936 24658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:21.853022 24655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:21.854548 24648 server_base.cc:1034] running on GCE node
I20260812 02:43:21.855433 24648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:21.856580 24648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:21.857852 24648 hybrid_clock.cc:648] HybridClock initialized: now 1786502601857803 us; error 44 us; skew 500 ppm
I20260812 02:43:21.858295 24648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:21 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502601, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:21.862344 24648 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:43:21.864620 24648 webserver.cc:460] Webserver started at http://127.17.196.194:42519/ using document root <none> and password file <none>
I20260812 02:43:21.865383 24648 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:21.865602 24648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:21.865921 24648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:21.869167 24648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "c68f0066ba0f476e91c4810eaed55806"
format_stamp: "Formatted at 2026-08-12 02:43:21 on dist-test-slave-ss4n"
server_key: "2b43c78c47951518f47091deb764f1ee"
server_key_iv: "ec7e42b2745ef10d05e8d34b1cabdf80"
server_key_version: "encryptionkey@0"
I20260812 02:43:21.870163 24648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "c68f0066ba0f476e91c4810eaed55806"
format_stamp: "Formatted at 2026-08-12 02:43:21 on dist-test-slave-ss4n"
server_key: "2b43c78c47951518f47091deb764f1ee"
server_key_iv: "ec7e42b2745ef10d05e8d34b1cabdf80"
server_key_version: "encryptionkey@0"
I20260812 02:43:21.875910 24648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 02:43:21.879860 24666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:21.881379 24648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:43:21.881623 24648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "c68f0066ba0f476e91c4810eaed55806"
format_stamp: "Formatted at 2026-08-12 02:43:21 on dist-test-slave-ss4n"
server_key: "2b43c78c47951518f47091deb764f1ee"
server_key_iv: "ec7e42b2745ef10d05e8d34b1cabdf80"
server_key_version: "encryptionkey@0"
I20260812 02:43:21.881841 24648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:21.900573 24648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:21.906098 24648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:21.906458 24648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:21.907699 24648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:21.909409 24648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:21.909575 24648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:21.909793 24648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:21.909853 24648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:21.953367 24648 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:42739
I20260812 02:43:21.953409 24791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:42739 every 8 connection(s)
I20260812 02:43:21.954941 24648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:21.958707 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 24648
I20260812 02:43:21.958936 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:43:21.959640 18195 external_mini_cluster.cc:1428] Setting key 0169eda66dbf3f32de5abbf49d4edbc4
Aug 12 02:43:21 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502601, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:21.979031 24794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:21.958106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:49325 (local address 127.17.196.254:44459)
0812 02:43:21.958644 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:21.958650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:21.960082 (+ 1432us) server_negotiation.cc:407] Connection header received
0812 02:43:21.962509 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:21.962516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:21.962625 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:21.962770 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:21.964627 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:21.965439 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:21.967271 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:21.967806 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:21.967970 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:21.972104 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:21.972110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:21.972113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:21.972153 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:21.975341 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:21.976237 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:21.976246 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:21.976251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:21.976414 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:21.976946 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:21.976953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:21.976957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:21.977289 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:21.977476 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:21.978318 (+ 842us) server_negotiation.cc:299] Negotiation successful
0812 02:43:21.978595 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":292,"threads_started":1}
I20260812 02:43:21.980199 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:21.958819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44459 (local address 127.17.196.194:49325)
0812 02:43:21.959856 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:21.959924 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:21.962130 (+ 2206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:21.963071 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:21.963080 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:21.963746 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:21.964382 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:21.964401 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:21.965664 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:21.965671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:21.967076 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:21.967090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:21.968100 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:21.968108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:21.968381 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:21.969222 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:21.969255 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:21.971802 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:21.975534 (+ 3732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:21.975546 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:21.975566 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:21.976087 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:21.976574 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:21.976579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:21.976584 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:21.976811 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:21.977446 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:21.977455 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:21.977900 (+ 445us) client_negotiation.cc:769] Sending connection context
0812 02:43:21.978232 (+ 332us) client_negotiation.cc:240] Negotiation successful
0812 02:43:21.978864 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":375,"threads_started":1}
I20260812 02:43:21.982087 24792 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44459
I20260812 02:43:21.982496 24792 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:21.983491 24792 heartbeater.cc:511] Master 127.17.196.254:44459 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
I20260812 02:43:21.985391 24429 ts_manager.cc:194] Registered new tserver with Master: c68f0066ba0f476e91c4810eaed55806 (127.17.196.194:42739)
I20260812 02:43:21.986472 24429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:49325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:43:22.021677 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44459
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:22.382462 24800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:22.382754 24800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:22.382851 24800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:22.390506 24800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:22.390645 24800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:22.390764 24800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:22.401086 24800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:22.402597 24800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:22.403937 24800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:43:22.410650 24641 heartbeater.cc:503] Master 127.17.196.254:44459 was elected leader, sending a full tablet report...
W20260812 02:43:22.418762 24805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:22.418726 24806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:22.418871 24808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:22.420022 24800 server_base.cc:1034] running on GCE node
I20260812 02:43:22.420727 24800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:22.421932 24800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:22.423218 24800 hybrid_clock.cc:648] HybridClock initialized: now 1786502602423164 us; error 44 us; skew 500 ppm
I20260812 02:43:22.423631 24800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:22.427441 24800 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:43:22.429072 24800 webserver.cc:460] Webserver started at http://127.17.196.195:41915/ using document root <none> and password file <none>
I20260812 02:43:22.429924 24800 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:22.430044 24800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:22.430361 24800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:22.434013 24800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "366d4f20e3454039942fa4a8edc5bbb2"
format_stamp: "Formatted at 2026-08-12 02:43:22 on dist-test-slave-ss4n"
server_key: "c381c209c1c8d977cdf77397fc1c7b7d"
server_key_iv: "79413e1c27301120e4a35ecb78dbc80e"
server_key_version: "encryptionkey@0"
I20260812 02:43:22.434911 24800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "366d4f20e3454039942fa4a8edc5bbb2"
format_stamp: "Formatted at 2026-08-12 02:43:22 on dist-test-slave-ss4n"
server_key: "c381c209c1c8d977cdf77397fc1c7b7d"
server_key_iv: "79413e1c27301120e4a35ecb78dbc80e"
server_key_version: "encryptionkey@0"
I20260812 02:43:22.441268 24800 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 02:43:22.445910 24818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:22.447719 24800 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:43:22.447923 24800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "366d4f20e3454039942fa4a8edc5bbb2"
format_stamp: "Formatted at 2026-08-12 02:43:22 on dist-test-slave-ss4n"
server_key: "c381c209c1c8d977cdf77397fc1c7b7d"
server_key_iv: "79413e1c27301120e4a35ecb78dbc80e"
server_key_version: "encryptionkey@0"
I20260812 02:43:22.448155 24800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:22.464918 24800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:22.470131 24800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:22.470409 24800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:22.471464 24800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:22.472875 24800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:22.472950 24800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:22.473034 24800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:22.473085 24800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:22.511327 24800 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:43107
I20260812 02:43:22.511361 24941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:43107 every 8 connection(s)
I20260812 02:43:22.512852 24800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:22.521057 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 24800
I20260812 02:43:22.521272 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:43:22.521852 18195 external_mini_cluster.cc:1428] Setting key e9abe823ebe2f35de7dd59bdd6365157
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:22.536226 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:22.515854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:38989 (local address 127.17.196.254:44459)
0812 02:43:22.516358 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.516364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.517444 (+ 1080us) server_negotiation.cc:407] Connection header received
0812 02:43:22.519375 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.519386 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.519482 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.519603 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.521613 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.522635 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.524552 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.525007 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.525163 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:22.529082 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:22.529089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:22.529093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:22.529149 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:22.532434 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.533255 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.533259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.533262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.533672 (+ 410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.534185 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.534191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.534195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.534590 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:22.534749 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:22.535561 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 02:43:22.535802 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":268,"threads_started":1}
I20260812 02:43:22.537448 24946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:22.516462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44459 (local address 127.17.196.195:38989)
0812 02:43:22.517259 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:22.517301 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:22.519042 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:22.519839 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:22.519848 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:22.520441 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:22.521359 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:22.521389 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.522809 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.522813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:22.524382 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:22.524396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.525267 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.525278 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:22.525617 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:22.526304 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:22.526328 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:22.528801 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:22.532631 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:22.532639 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:22.532655 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:22.533115 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:22.533796 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:22.533799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:22.533802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:22.533996 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:22.534721 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:22.534728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:22.535080 (+ 352us) client_negotiation.cc:769] Sending connection context
0812 02:43:22.535319 (+ 239us) client_negotiation.cc:240] Negotiation successful
0812 02:43:22.535920 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":295,"threads_started":1}
I20260812 02:43:22.538923 24942 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44459
I20260812 02:43:22.539316 24942 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:22.540131 24942 heartbeater.cc:511] Master 127.17.196.254:44459 requested a full tablet report, sending...
I20260812 02:43:22.541868 24429 ts_manager.cc:194] Registered new tserver with Master: 366d4f20e3454039942fa4a8edc5bbb2 (127.17.196.195:43107)
I20260812 02:43:22.542768 24429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:38989
I20260812 02:43:22.551251 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/test-admin.keytab.
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:22.588280 18195 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 02:43:22.594223 24952 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:22.591343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.17.196.193:39093)
0812 02:43:22.592095 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.592103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.592132 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:43:22.592198 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.592206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.592426 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.592584 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.593700 (+ 1116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54144: BlockingRecv error: recv got EOF from 127.0.0.1:54144 (error 108)
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":398,"threads_started":1}
W20260812 02:43:22.594700 18195 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5cc)
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:22.596586 18195 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:22.598321 18195 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:43:22.616804 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:22.600438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.17.196.193:39093)
0812 02:43:22.600662 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.600667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.600765 (+ 98us) server_negotiation.cc:407] Connection header received
0812 02:43:22.600942 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.600946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.601015 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.601111 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.602394 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.603421 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.604898 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.605685 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.605812 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:22.610154 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:22.610189 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:22.610211 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:22.610262 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:22.613819 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.614715 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.614723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.614725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.614860 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.615264 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.615267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.615269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.615658 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:22.615924 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:22.616288 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 02:43:22.616503 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 02:43:22.623556 24952 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:22.621557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54152 (local address 127.17.196.193:39093)
0812 02:43:22.621794 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.621800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.622023 (+ 223us) server_negotiation.cc:407] Connection header received
0812 02:43:22.622172 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.622176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.622256 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.622345 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.623267 (+ 922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54152: BlockingRecv error: recv got EOF from 127.0.0.1:54152 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 02:43:22.623550 18195 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5cc' of type FILE
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:22.625172 18195 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:22.626585 18195 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:43:22.643651 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:22.627992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54154 (local address 127.17.196.193:39093)
0812 02:43:22.628219 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.628224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.628509 (+ 285us) server_negotiation.cc:407] Connection header received
0812 02:43:22.628595 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.628599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.628665 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.628770 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.629909 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.630845 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.632114 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.632593 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.632701 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:22.636689 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:22.636695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:22.636699 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:22.636740 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:22.639986 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.640812 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.640816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.640819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.640977 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.641620 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.641626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.641629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.642018 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:22.642261 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:22.642706 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 02:43:22.642943 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 02:43:22.649467 24952 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:22.647014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.17.196.193:39093)
0812 02:43:22.647390 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.647397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.647704 (+ 307us) server_negotiation.cc:407] Connection header received
0812 02:43:22.647950 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.647961 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.648047 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.648179 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.649133 (+ 954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54168: BlockingRecv error: recv got EOF from 127.0.0.1:54168 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
W20260812 02:43:22.649773 18195 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502547576936-18195-0/krb5kdc/krb5cc' of type FILE
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:22.651587 18195 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:22.653142 18195 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:43:22.669792 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:22.654993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.17.196.193:39093)
0812 02:43:22.655310 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.655317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.655635 (+ 318us) server_negotiation.cc:407] Connection header received
0812 02:43:22.655792 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.655796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.655872 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.656043 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.657034 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.657865 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.659288 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.659951 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.660063 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:22.664160 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:22.664165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:22.664169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:22.664207 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:22.666940 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.667722 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.667726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.667729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.667890 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.668290 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.668293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.668295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.668637 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:22.668873 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:22.669172 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 02:43:22.669392 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260812 02:43:22.671845 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 24498
I20260812 02:43:22.685616 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:22.672765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.17.196.193:39093)
0812 02:43:22.672995 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.673001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.673220 (+ 219us) server_negotiation.cc:407] Connection header received
0812 02:43:22.673432 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.673438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.673554 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.673682 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.674742 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.675445 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.676750 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.677212 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.677360 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:22.679726 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:22.679731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:22.679735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:22.679768 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:22.682395 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.683099 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.683105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.683109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.683279 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.683815 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.683824 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.683828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.684197 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:22.684610 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:22.684880 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 02:43:22.685145 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:43:22.686347 24636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:22.779304 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 24498
I20260812 02:43:22.800400 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 24648
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:43:22.818070 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:22.801435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50282 (local address 127.17.196.194:42739)
0812 02:43:22.802065 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.802072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.802099 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:43:22.802186 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.802192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.802413 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.802616 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.803684 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.804700 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.806090 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.806693 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.806853 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:22.811324 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:22.811338 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:22.811355 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:22.811396 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:22.814792 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.815662 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.815671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.815673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.815819 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.816291 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.816295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.816298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.816678 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:22.817032 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:22.817408 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 02:43:22.817752 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":296,"threads_started":1}
I20260812 02:43:22.819639 24787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:22.912993 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 24648
I20260812 02:43:22.932699 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 24800
Aug 12 02:43:22 dist-test-slave-ss4n krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:43:22.952239 24969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:22.933953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.17.196.195:43107)
0812 02:43:22.934867 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:22.934875 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:22.934908 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:43:22.934972 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:22.934980 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:22.935253 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:22.935396 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:22.936607 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.937857 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.939362 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:22.939981 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:22.940113 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:22.945194 (+ 5081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:22.945211 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:22.945228 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:22.945275 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:22.948779 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.949718 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.949727 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.949730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.949886 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:22.950396 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:22.950402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:22.950405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:22.950826 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:22.951219 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:22.951534 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 02:43:22.951842 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":474,"threads_started":1}
I20260812 02:43:22.953824 24935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:23.041217 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 24800
I20260812 02:43:23.062151 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 24396
I20260812 02:43:23.063719 24462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:23.171216 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 24396
2026-08-12T02:43:23Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4481 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:43:23 dist-test-slave-ss4n krb5kdc[24975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:43:23 dist-test-slave-ss4n krb5kdc[24975](info): set up 2 sockets
Aug 12 02:43:23 dist-test-slave-ss4n krb5kdc[24975](info): commencing operation
krb5kdc: starting...
W20260812 02:43:24.425863 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:43:24 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:43:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.254.keytab.
Entry for principal oryx/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.254.keytab.
I20260812 02:43:24.711525 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:34567
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.254.keytab
--principal=oryx/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:34567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:25.081637 24988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:25.081967 24988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:25.082190 24988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:25.089953 24988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:25.090102 24988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:25.090183 24988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:25.090274 24988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:25.090317 24988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:25.101122 24988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:34567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.254.keytab
--principal=oryx/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:34567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:25.102662 24988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:25.104146 24988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:25.116686 24994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:25.116709 24996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:25.117030 24993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:25.118738 24988 server_base.cc:1034] running on GCE node
I20260812 02:43:25.119638 24988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:25.121891 24988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:25.123157 24988 hybrid_clock.cc:648] HybridClock initialized: now 1786502605123021 us; error 120 us; skew 500 ppm
I20260812 02:43:25.123654 24988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:25 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502605, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:25.128322 24988 init.cc:377] Logged in from keytab as oryx/127.17.196.254@KRBTEST.COM (short username oryx)
I20260812 02:43:25.130153 24988 webserver.cc:460] Webserver started at http://127.17.196.254:39815/ using document root <none> and password file <none>
I20260812 02:43:25.131125 24988 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:25.131283 24988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:25.131701 24988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:25.136359 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "2dce354ed6604f9dba212f727ac21f28"
format_stamp: "Formatted at 2026-08-12 02:43:25 on dist-test-slave-ss4n"
server_key: "de247e9ab9f2f8879b3f20e00f669a8a"
server_key_iv: "0f0bf4099887388f62b6ee8638691235"
server_key_version: "encryptionkey@0"
I20260812 02:43:25.137507 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "2dce354ed6604f9dba212f727ac21f28"
format_stamp: "Formatted at 2026-08-12 02:43:25 on dist-test-slave-ss4n"
server_key: "de247e9ab9f2f8879b3f20e00f669a8a"
server_key_iv: "0f0bf4099887388f62b6ee8638691235"
server_key_version: "encryptionkey@0"
I20260812 02:43:25.146344 24988 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260812 02:43:25.152160 25003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:25.154352 24988 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 02:43:25.154565 24988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "2dce354ed6604f9dba212f727ac21f28"
format_stamp: "Formatted at 2026-08-12 02:43:25 on dist-test-slave-ss4n"
server_key: "de247e9ab9f2f8879b3f20e00f669a8a"
server_key_iv: "0f0bf4099887388f62b6ee8638691235"
server_key_version: "encryptionkey@0"
I20260812 02:43:25.154826 24988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:25.174876 24988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:25.179942 24988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:25.180282 24988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:25.202239 24988 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:34567
I20260812 02:43:25.202294 25061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:34567 every 8 connection(s)
I20260812 02:43:25.204075 24988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:25.208320 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:25.212172 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 24988
I20260812 02:43:25.212421 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:25.213254 18195 external_mini_cluster.cc:1428] Setting key f40e54b093d8d2adb1150aca254cb0a0
I20260812 02:43:25.220077 25062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: Bootstrap starting.
I20260812 02:43:25.223477 25062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:25.224575 25062 log.cc:824] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:25.228853 25062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: No bootstrap required, opened a new log
Aug 12 02:43:25 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:25.237649 25062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } }
I20260812 02:43:25.238150 25062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:25.238301 25062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dce354ed6604f9dba212f727ac21f28, State: Initialized, Role: FOLLOWER
I20260812 02:43:25.239003 25062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } }
I20260812 02:43:25.239293 25062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:25.239467 25062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:25.239671 25062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:25.239670 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:25.215572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.17.196.254:34567)
0812 02:43:25.216450 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:25.216464 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:25.216514 (+ 50us) server_negotiation.cc:407] Connection header received
0812 02:43:25.218972 (+ 2458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:25.218999 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:25.219446 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:25.219907 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:25.221686 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:25.223178 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:25.224681 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:25.226022 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:25.226242 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:25.231407 (+ 5165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:25.231418 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:25.231435 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:25.231503 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:25.234935 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:25.235727 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:25.235745 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:25.235753 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:25.235949 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:25.236419 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:25.236424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:25.236427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:25.237039 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:25.237385 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:25.237887 (+ 502us) server_negotiation.cc:299] Negotiation successful
0812 02:43:25.238444 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":367,"threads_started":1}
I20260812 02:43:25.242946 25062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } }
I20260812 02:43:25.243600 25062 leader_election.cc:302] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dce354ed6604f9dba212f727ac21f28; no voters:
I20260812 02:43:25.244357 25062 leader_election.cc:288] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:25.244539 25067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:25.245458 25067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 1 LEADER]: Becoming Leader. State: Replica: 2dce354ed6604f9dba212f727ac21f28, State: Running, Role: LEADER
I20260812 02:43:25.246155 25067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } }
I20260812 02:43:25.246855 25062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:25.249512 25069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dce354ed6604f9dba212f727ac21f28. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } }
I20260812 02:43:25.249627 25068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } }
I20260812 02:43:25.250033 25069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:25.250095 25068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:25.251130 25073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:25.257565 25073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:25.269026 25073 catalog_manager.cc:1347] Generated new cluster ID: 84bd3a9d86e94265ae1b366d3829c551
I20260812 02:43:25.269207 25073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:25.308483 25073 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:25.310719 25073 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:25.319427 25073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: Generated new TSK 0
I20260812 02:43:25.320674 25073 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab.
Entry for principal oryx/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab.
I20260812 02:43:25.408123 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:34567
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab
--principal=oryx/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:25.808262 25090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:25.808614 25090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:25.808781 25090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:25.817062 25090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:25.817241 25090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:25.817463 25090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:25.829977 25090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab
--principal=oryx/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:34567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:25.831485 25090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:25.833012 25090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:25.848780 25099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:25.848692 25101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:25.849258 25097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:25.850013 25090 server_base.cc:1034] running on GCE node
I20260812 02:43:25.850747 25090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:25.851928 25090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:25.853225 25090 hybrid_clock.cc:648] HybridClock initialized: now 1786502605853136 us; error 54 us; skew 500 ppm
I20260812 02:43:25.853669 25090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:25 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502605, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:25.862594 25090 init.cc:377] Logged in from keytab as oryx/127.17.196.193@KRBTEST.COM (short username oryx)
I20260812 02:43:25.864343 25090 webserver.cc:460] Webserver started at http://127.17.196.193:40129/ using document root <none> and password file <none>
I20260812 02:43:25.865082 25090 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:25.865193 25090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:25.865443 25090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:25.868782 25090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "f8a58be1415e4d268335e44dec2c1baf"
format_stamp: "Formatted at 2026-08-12 02:43:25 on dist-test-slave-ss4n"
server_key: "67432f6b7a06fdb3ec4b80f2b346a1d2"
server_key_iv: "15453256bc9037e3639992254e16ead5"
server_key_version: "encryptionkey@0"
I20260812 02:43:25.869845 25090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "f8a58be1415e4d268335e44dec2c1baf"
format_stamp: "Formatted at 2026-08-12 02:43:25 on dist-test-slave-ss4n"
server_key: "67432f6b7a06fdb3ec4b80f2b346a1d2"
server_key_iv: "15453256bc9037e3639992254e16ead5"
server_key_version: "encryptionkey@0"
I20260812 02:43:25.875404 25090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:43:25.879312 25108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:25.881075 25090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:43:25.881294 25090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "f8a58be1415e4d268335e44dec2c1baf"
format_stamp: "Formatted at 2026-08-12 02:43:25 on dist-test-slave-ss4n"
server_key: "67432f6b7a06fdb3ec4b80f2b346a1d2"
server_key_iv: "15453256bc9037e3639992254e16ead5"
server_key_version: "encryptionkey@0"
I20260812 02:43:25.881609 25090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:25.899199 25090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:25.904737 25090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:25.905121 25090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:25.906466 25090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:25.908128 25090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:25.908213 25090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:25.908386 25090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:25.908463 25090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:25.952145 25090 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:34391
I20260812 02:43:25.952209 25234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:34391 every 8 connection(s)
I20260812 02:43:25.953917 25090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:25.959606 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 25090
I20260812 02:43:25.959860 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:43:25.960832 18195 external_mini_cluster.cc:1428] Setting key 4d690541502cd799c661aad8996c8bf8
Aug 12 02:43:25 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502605, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:25.979629 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:25.956809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:43449 (local address 127.17.196.254:34567)
0812 02:43:25.957430 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:25.957438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:25.958698 (+ 1260us) server_negotiation.cc:407] Connection header received
0812 02:43:25.961784 (+ 3086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:25.961793 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:25.961896 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:25.962007 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:25.964309 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:25.965094 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:25.967998 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:25.968400 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:25.968513 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:25.973428 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:25.973434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:25.973437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:25.973528 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:25.976274 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:25.977082 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:25.977089 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:25.977095 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:25.977229 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:25.977762 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:25.977767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:25.977770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:25.978056 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:25.978215 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:25.978972 (+ 757us) server_negotiation.cc:299] Negotiation successful
0812 02:43:25.979223 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":318,"threads_started":1}
I20260812 02:43:25.980748 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:25.957570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.193:43449)
0812 02:43:25.958477 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:25.958537 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:25.961363 (+ 2826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:25.962359 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:25.962369 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:25.963143 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:25.964081 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:25.964116 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:25.965287 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:25.965297 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:25.967801 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:25.967824 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:25.968677 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:25.968686 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:25.969075 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:25.970056 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:25.970097 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:25.973164 (+ 3067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:25.976446 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:25.976456 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:25.976477 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:25.976940 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:25.977350 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:25.977355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:25.977359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:25.977632 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:25.978192 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:25.978201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:25.978580 (+ 379us) client_negotiation.cc:769] Sending connection context
0812 02:43:25.978853 (+ 273us) client_negotiation.cc:240] Negotiation successful
0812 02:43:25.979481 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":357,"threads_started":1}
I20260812 02:43:25.982810 25235 heartbeater.cc:347] Connected to a master server at 127.17.196.254:34567
I20260812 02:43:25.983162 25235 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:25.983989 25235 heartbeater.cc:511] Master 127.17.196.254:34567 requested a full tablet report, sending...
I20260812 02:43:25.986508 25024 ts_manager.cc:194] Registered new tserver with Master: f8a58be1415e4d268335e44dec2c1baf (127.17.196.193:34391)
I20260812 02:43:25.988118 25024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:43449
WARNING: no policy specified for oryx/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab.
Entry for principal oryx/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab.
I20260812 02:43:26.029188 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:34567
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab
--principal=oryx/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:26.390180 25242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:26.390617 25242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:26.390805 25242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:26.399365 25242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:26.399503 25242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:26.399648 25242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:26.410064 25242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab
--principal=oryx/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:34567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:26.411369 25242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:26.412701 25242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:26.426892 25250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:26.427183 25247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:26.427256 25248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:26.428171 25242 server_base.cc:1034] running on GCE node
I20260812 02:43:26.428879 25242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:26.430023 25242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:26.431277 25242 hybrid_clock.cc:648] HybridClock initialized: now 1786502606431231 us; error 30 us; skew 500 ppm
I20260812 02:43:26.431677 25242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:26 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502606, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:26.435320 25242 init.cc:377] Logged in from keytab as oryx/127.17.196.194@KRBTEST.COM (short username oryx)
I20260812 02:43:26.436954 25242 webserver.cc:460] Webserver started at http://127.17.196.194:34895/ using document root <none> and password file <none>
I20260812 02:43:26.437876 25242 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:26.437986 25242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:26.438251 25242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:26.441273 25242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "4af9696aa3b441cabaf65f148ff973f2"
format_stamp: "Formatted at 2026-08-12 02:43:26 on dist-test-slave-ss4n"
server_key: "8d056b20375dbaee3e3f616ea135ab1d"
server_key_iv: "d43fc63da87be53a8c981b064ce492a3"
server_key_version: "encryptionkey@0"
I20260812 02:43:26.442203 25242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "4af9696aa3b441cabaf65f148ff973f2"
format_stamp: "Formatted at 2026-08-12 02:43:26 on dist-test-slave-ss4n"
server_key: "8d056b20375dbaee3e3f616ea135ab1d"
server_key_iv: "d43fc63da87be53a8c981b064ce492a3"
server_key_version: "encryptionkey@0"
I20260812 02:43:26.448330 25242 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:43:26.452129 25257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:26.453939 25242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:43:26.454146 25242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "4af9696aa3b441cabaf65f148ff973f2"
format_stamp: "Formatted at 2026-08-12 02:43:26 on dist-test-slave-ss4n"
server_key: "8d056b20375dbaee3e3f616ea135ab1d"
server_key_iv: "d43fc63da87be53a8c981b064ce492a3"
server_key_version: "encryptionkey@0"
I20260812 02:43:26.454334 25242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:26.472489 25242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:26.477144 25242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:26.477401 25242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:26.478564 25242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:26.480108 25242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:26.480234 25242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:26.480408 25242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:26.480491 25242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:26.520985 25242 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:42369
I20260812 02:43:26.521040 25381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:42369 every 8 connection(s)
I20260812 02:43:26.522585 25242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:26.529266 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 25242
I20260812 02:43:26.529485 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:43:26.530073 18195 external_mini_cluster.cc:1428] Setting key a72f410a1d7790c414154b448b1f8137
Aug 12 02:43:26 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502606, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:26.545252 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:26.525686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:54171 (local address 127.17.196.254:34567)
0812 02:43:26.526178 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:26.526183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:26.527149 (+ 966us) server_negotiation.cc:407] Connection header received
0812 02:43:26.529074 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:26.529080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:26.529180 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:26.529313 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:26.531289 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:26.532202 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:26.533915 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:26.534386 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:26.534539 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:26.538414 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:26.538419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:26.538423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:26.538458 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:26.541308 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:26.542196 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:26.542202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:26.542206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:26.542335 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:26.542965 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:26.542969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:26.542971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:26.543288 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:26.543425 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:26.544690 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0812 02:43:26.544932 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":278,"threads_started":1}
I20260812 02:43:26.546664 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:26.526203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.194:54171)
0812 02:43:26.526958 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:26.527011 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:26.528749 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:26.529686 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:26.529696 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:26.530333 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:26.531093 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:26.531116 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:26.532392 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:26.532396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:26.533759 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:26.533772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:26.534658 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:26.534667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:26.534904 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:26.535692 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:26.535721 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:26.538129 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:26.541535 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:26.541547 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:26.541566 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:26.542027 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:26.542495 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:26.542504 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:26.542507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:26.542800 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:26.543442 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:26.543454 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:26.543948 (+ 494us) client_negotiation.cc:769] Sending connection context
0812 02:43:26.544282 (+ 334us) client_negotiation.cc:240] Negotiation successful
0812 02:43:26.544932 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":304,"threads_started":1}
I20260812 02:43:26.548831 25382 heartbeater.cc:347] Connected to a master server at 127.17.196.254:34567
I20260812 02:43:26.549229 25382 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:26.550614 25382 heartbeater.cc:511] Master 127.17.196.254:34567 requested a full tablet report, sending...
I20260812 02:43:26.552839 25024 ts_manager.cc:194] Registered new tserver with Master: 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194:42369)
I20260812 02:43:26.554574 25024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:54171
WARNING: no policy specified for oryx/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab.
Entry for principal oryx/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab.
I20260812 02:43:26.598937 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:34567
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab
--principal=oryx/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:26.976522 25388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:26.976913 25388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:26.977021 25388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:26.985011 25388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:26.985153 25388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:26.985271 25388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:26.991544 25235 heartbeater.cc:503] Master 127.17.196.254:34567 was elected leader, sending a full tablet report...
I20260812 02:43:26.996208 25388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab
--principal=oryx/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:34567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:26.997785 25388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:26.999200 25388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:27.015784 25394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:27.015806 25397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:27.015806 25395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:27.016708 25388 server_base.cc:1034] running on GCE node
I20260812 02:43:27.017540 25388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:27.018676 25388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:27.019976 25388 hybrid_clock.cc:648] HybridClock initialized: now 1786502607019905 us; error 56 us; skew 500 ppm
I20260812 02:43:27.020406 25388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:27 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502607, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:27.024470 25388 init.cc:377] Logged in from keytab as oryx/127.17.196.195@KRBTEST.COM (short username oryx)
I20260812 02:43:27.026386 25388 webserver.cc:460] Webserver started at http://127.17.196.195:42847/ using document root <none> and password file <none>
I20260812 02:43:27.027133 25388 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:27.027252 25388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:27.027542 25388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:27.030926 25388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "8b1cd287a35546828e6a22f690827612"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "4f49ebede19857bd784533c0bd4e626b"
server_key_iv: "2f02d7143f3e9597498368d3a7a81bd6"
server_key_version: "encryptionkey@0"
I20260812 02:43:27.031795 25388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "8b1cd287a35546828e6a22f690827612"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "4f49ebede19857bd784533c0bd4e626b"
server_key_iv: "2f02d7143f3e9597498368d3a7a81bd6"
server_key_version: "encryptionkey@0"
I20260812 02:43:27.037846 25388 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 02:43:27.041836 25405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:27.043593 25388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:43:27.043758 25388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "8b1cd287a35546828e6a22f690827612"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "4f49ebede19857bd784533c0bd4e626b"
server_key_iv: "2f02d7143f3e9597498368d3a7a81bd6"
server_key_version: "encryptionkey@0"
I20260812 02:43:27.044051 25388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:27.063459 25388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:27.069182 25388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:27.069738 25388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:27.070931 25388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:27.072582 25388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:27.072710 25388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:27.072907 25388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:27.072997 25388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:27.113046 25388 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:43795
I20260812 02:43:27.113150 25530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:43795 every 8 connection(s)
I20260812 02:43:27.114634 25388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:27.120137 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 25388
I20260812 02:43:27.120407 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:43:27.120960 18195 external_mini_cluster.cc:1428] Setting key 6563c1c7cbb27d97526f19ea97644841
Aug 12 02:43:27 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502607, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:27.138471 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.117577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:59339 (local address 127.17.196.254:34567)
0812 02:43:27.118107 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:27.118112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:27.119033 (+ 921us) server_negotiation.cc:407] Connection header received
0812 02:43:27.121109 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:27.121115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:27.121223 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:27.121358 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:27.123281 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.124678 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.126378 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.126826 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.126933 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.131017 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:27.131022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:27.131026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:27.131062 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:27.134262 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.135176 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.135181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.135183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.135398 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.136092 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.136095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.136097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.136585 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:27.136770 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:27.137779 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0812 02:43:27.138030 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":265,"threads_started":1}
I20260812 02:43:27.139585 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.118075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.195:59339)
0812 02:43:27.118869 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:27.118913 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:27.120806 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:27.121648 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:27.121657 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:27.122421 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:27.123040 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.123058 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.124862 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.124867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.126227 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.126240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.127027 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.127032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.127275 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.128082 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:27.128107 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:27.130744 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:27.134435 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.134443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.134458 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.135005 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.135653 (+ 648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.135659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.135662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.135942 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.136779 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:27.136788 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:27.137257 (+ 469us) client_negotiation.cc:769] Sending connection context
0812 02:43:27.137666 (+ 409us) client_negotiation.cc:240] Negotiation successful
0812 02:43:27.138252 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":372,"threads_started":1}
I20260812 02:43:27.141299 25532 heartbeater.cc:347] Connected to a master server at 127.17.196.254:34567
I20260812 02:43:27.141772 25532 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:27.142601 25532 heartbeater.cc:511] Master 127.17.196.254:34567 requested a full tablet report, sending...
I20260812 02:43:27.144342 25024 ts_manager.cc:194] Registered new tserver with Master: 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795)
I20260812 02:43:27.145299 25024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:59339
I20260812 02:43:27.150456 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.196.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.253.keytab.
Entry for principal oryx/127.17.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.253.keytab.
I20260812 02:43:27.214187 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.253:46567
--webserver_interface=127.17.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.253.keytab
--principal=oryx/127.17.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:34567,127.17.196.253:46567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:43:27.558089 25382 heartbeater.cc:503] Master 127.17.196.254:34567 was elected leader, sending a full tablet report...
W20260812 02:43:27.577467 25538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:27.577854 25538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:27.578044 25538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:27.585798 25538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:27.585916 25538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:27.586002 25538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:27.586109 25538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:27.586171 25538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:27.596522 25538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:34567,127.17.196.253:46567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.253.keytab
--principal=oryx/127.17.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.253:46567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
W20260812 02:43:27.596964 25538 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.196.254:34567,127.17.196.253:46567'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 02:43:27.598166 25538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:27.599622 25538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:27.613386 25545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:27.613364 25548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:27.613309 25546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:27.614464 25538 server_base.cc:1034] running on GCE node
I20260812 02:43:27.615373 25538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:27.617132 25538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:27.618422 25538 hybrid_clock.cc:648] HybridClock initialized: now 1786502607618341 us; error 128 us; skew 500 ppm
I20260812 02:43:27.618819 25538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:27 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502607, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:27.622601 25538 init.cc:377] Logged in from keytab as oryx/127.17.196.253@KRBTEST.COM (short username oryx)
I20260812 02:43:27.624424 25538 webserver.cc:460] Webserver started at http://127.17.196.253:40997/ using document root <none> and password file <none>
I20260812 02:43:27.625224 25538 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:27.625371 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:27.625782 25538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:27.628961 25538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data/instance:
uuid: "b1f1396065e740869a40a86ff99bf02f"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "33ff4389ba7e55ba1acc4935e1682c69"
server_key_iv: "756a37d4777ca858127561a3975faa61"
server_key_version: "encryptionkey@0"
I20260812 02:43:27.629878 25538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal/instance:
uuid: "b1f1396065e740869a40a86ff99bf02f"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "33ff4389ba7e55ba1acc4935e1682c69"
server_key_iv: "756a37d4777ca858127561a3975faa61"
server_key_version: "encryptionkey@0"
I20260812 02:43:27.635635 25538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:43:27.639451 25555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:27.641129 25538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:43:27.641347 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
uuid: "b1f1396065e740869a40a86ff99bf02f"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "33ff4389ba7e55ba1acc4935e1682c69"
server_key_iv: "756a37d4777ca858127561a3975faa61"
server_key_version: "encryptionkey@0"
I20260812 02:43:27.641645 25538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:27.662358 25538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:27.667289 25538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:27.667654 25538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:27.690065 25538 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.253:46567
I20260812 02:43:27.690117 25614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.253:46567 every 8 connection(s)
I20260812 02:43:27.691562 25538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
I20260812 02:43:27.692749 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 25538
I20260812 02:43:27.697110 25615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:27.707108 25615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:43:27 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.253@KRBTEST.COM
Aug 12 02:43:27 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502607, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.253@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:27.722064 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.695656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.17.196.253:46567)
0812 02:43:27.696753 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:27.696771 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:27.697303 (+ 532us) server_negotiation.cc:407] Connection header received
0812 02:43:27.699167 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:27.699191 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:27.699580 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:27.699969 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:27.702205 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.703659 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.705748 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.706498 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.706647 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.712586 (+ 5939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:27.712595 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:27.712612 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:27.712706 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:27.716899 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.717801 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.717808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.717815 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.718060 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.718526 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.718531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.718535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.719338 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:27.719716 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:27.720153 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 02:43:27.720636 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":445,"threads_started":1}
I20260812 02:43:27.728787 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.711097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:48210)
0812 02:43:27.712493 (+ 1396us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:27.712554 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:27.712751 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:27.713253 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:27.713263 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:27.714277 (+ 1014us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:27.714687 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.714700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.716289 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.716295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.717873 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.717888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.718811 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.718816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.718951 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.719692 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:27.719720 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:27.722529 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:27.726197 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.726207 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.726212 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.726634 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.727102 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.727106 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.727108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.727256 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.727836 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:27.727842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:27.728004 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 02:43:27.728178 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 02:43:27.728437 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":897,"threads_started":1}
I20260812 02:43:27.728935 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.711085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48210 (local address 127.17.196.254:34567)
0812 02:43:27.711768 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:27.711778 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:27.712688 (+ 910us) server_negotiation.cc:407] Connection header received
0812 02:43:27.712913 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:27.712919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:27.713022 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:27.713152 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:27.714905 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.716098 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.718038 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.718620 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.718753 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.722768 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:27.722773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:27.722776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:27.722821 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:27.726056 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.726847 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.726851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.726854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.726992 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.727370 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.727373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.727375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.727718 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:27.727840 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:27.728302 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 02:43:27.728486 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":285,"threads_started":1}
I20260812 02:43:27.730870 25615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:43:27 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502607, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.253@KRBTEST.COM for oryx/127.17.196.253@KRBTEST.COM
I20260812 02:43:27.743954 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.731633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.17.196.253:46567)
0812 02:43:27.731826 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:27.731832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:27.731925 (+ 93us) server_negotiation.cc:407] Connection header received
0812 02:43:27.732108 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:27.732113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:27.732186 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:27.732293 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:27.732982 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.733807 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.734991 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.735414 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.735562 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.738690 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:27.738695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:27.738698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:27.738727 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:27.741221 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.742089 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.742093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.742095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.742252 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.742643 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.742647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.742649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.742938 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:27.743050 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:27.743471 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 02:43:27.743670 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:43:27.744302 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.731614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.0.0.1:42712)
0812 02:43:27.731826 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:27.731846 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:27.731988 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:27.732309 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:27.732313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:27.732711 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:27.732844 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.732853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.733957 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.733962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.734833 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.734844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.735576 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.735582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.735767 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.736314 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:27.736331 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:27.738491 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:27.741409 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.741414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.741418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.741969 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.742370 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.742374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.742376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.742530 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.743086 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:27.743090 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:27.743240 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 02:43:27.743376 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 02:43:27.743872 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":134}
I20260812 02:43:27.750476 25615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Bootstrap starting.
I20260812 02:43:27.754112 25615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:27.755203 25615 log.cc:824] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:27.758725 25615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: No bootstrap required, opened a new log
I20260812 02:43:27.767096 25615 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } }
I20260812 02:43:27.767452 25615 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:27.767537 25615 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f1396065e740869a40a86ff99bf02f, State: Initialized, Role: FOLLOWER
I20260812 02:43:27.768141 25615 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } }
I20260812 02:43:27.769052 25623 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } } }
I20260812 02:43:27.769452 25623 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:43:27.770277 25615 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: configured and running, proceeding with master startup.
W20260812 02:43:27.782226 25635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:43:27.782492 25635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:43:27.783648 25538 master_runner.cc:428] Detected that this master b1f1396065e740869a40a86ff99bf02f is joining an existing cluster
I20260812 02:43:27.783763 25538 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.196.253:46567
I20260812 02:43:27.807925 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.789188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.17.196.254:34567)
0812 02:43:27.789435 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:27.789443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:27.789467 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:43:27.789579 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:27.789586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:27.789689 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:27.789812 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:27.791635 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.792914 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.795670 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.796649 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.796785 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.800036 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:27.800045 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:27.800049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:27.800095 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:27.804183 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.805294 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.805303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.805307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.805517 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.806045 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.806053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.806057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.806452 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:27.806915 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:27.807284 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 02:43:27.807540 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 02:43:27.809583 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.788107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:48216)
0812 02:43:27.788794 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:27.788820 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:27.788965 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:27.789880 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:27.789888 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:27.790841 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:27.791406 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.791425 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.793079 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.793088 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.795434 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.795455 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.797204 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.797213 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.797507 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.798385 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:27.798419 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:27.799766 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:27.804373 (+ 4607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.804379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.804384 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.805108 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.805708 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.805713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.805716 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.805893 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.806693 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:27.806698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:27.806966 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 02:43:27.807175 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 02:43:27.807540 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":1231,"threads_started":1}
WARNING: no policy specified for oryx/127.17.196.252@KRBTEST.COM; defaulting to no policy
I20260812 02:43:27.818099 25646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.788823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.0.0.1:42728)
0812 02:43:27.799151 (+ 10328us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:27.799171 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:27.799615 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:27.800100 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:27.800105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:27.800790 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:27.801097 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.801113 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.802758 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.802764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.804430 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.804450 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.805995 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.806003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.806207 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.808421 (+ 2214us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:27.808440 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:27.809325 (+ 885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:27.814353 (+ 5028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.814360 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.814364 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.815011 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.815510 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.815514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.815518 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.815899 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.816852 (+ 953us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:27.816856 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:27.817009 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 02:43:27.817170 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 02:43:27.817561 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10163,"thread_start_us":10441,"threads_started":1}
I20260812 02:43:27.818547 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.789436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.17.196.253:46567)
0812 02:43:27.789730 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:27.789734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:27.799259 (+ 9525us) server_negotiation.cc:407] Connection header received
0812 02:43:27.799766 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:27.799772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:27.799845 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:27.799912 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:27.801287 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.802508 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.804579 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.805553 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.805659 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.809561 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:27.809569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:27.809576 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:27.809614 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:27.813958 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.815172 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.815177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.815180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.815348 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.816074 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.816079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.816083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.816635 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:27.816790 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:27.817371 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 02:43:27.817748 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":201}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.252@KRBTEST.COM" created.
Aug 12 02:43:27 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502605, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.254@KRBTEST.COM for oryx/127.17.196.253@KRBTEST.COM
I20260812 02:43:27.839182 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.817952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.0.0.1:42732)
0812 02:43:27.820476 (+ 2524us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:27.820537 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:27.820740 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:27.821392 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:27.821401 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:27.822416 (+ 1015us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:27.822758 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.822775 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.824729 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.824736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.826572 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.826591 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.827846 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.827854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.828086 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.828962 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:27.829000 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:27.831888 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:27.835402 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.835416 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.835424 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.836098 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.836647 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.836653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.836658 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.836853 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.837847 (+ 994us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:27.837857 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:27.838093 (+ 236us) client_negotiation.cc:769] Sending connection context
0812 02:43:27.838315 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 02:43:27.838704 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2239,"thread_start_us":2052,"threads_started":1}
I20260812 02:43:27.839458 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.818661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.17.196.253:46567)
0812 02:43:27.819501 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:27.819510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:27.820665 (+ 1155us) server_negotiation.cc:407] Connection header received
0812 02:43:27.821040 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:27.821046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:27.821151 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:27.821276 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:27.822973 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.824509 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.826750 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.827637 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.827790 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.832149 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:27.832155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:27.832158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:27.832190 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:27.835247 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.836278 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.836290 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.836293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.836475 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.837031 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.837035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.837038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.837680 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:27.837866 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:27.838534 (+ 668us) server_negotiation.cc:299] Negotiation successful
0812 02:43:27.838827 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":433,"threads_started":1}
I20260812 02:43:27.841702 25024 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.196.253:46567: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b1f1396065e740869a40a86ff99bf02f"
member_type: NON_VOTER
last_known_addr {
host: "127.17.196.253"
port: 46567
}
attrs {
promote: true
}
}
dest_uuid: "2dce354ed6604f9dba212f727ac21f28"
cas_config_opid_index: -1
I20260812 02:43:27.842824 25024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } }
I20260812 02:43:27.852995 25652 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b1f1396065e740869a40a86ff99bf02f (127.17.196.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } }
I20260812 02:43:27.854629 25587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:27.858208 25652 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.196.253:46567
I20260812 02:43:27.859058 25651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } }
I20260812 02:43:27.859386 25651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:27.859498 25538 master.cc:537] Master@127.17.196.253:46567 shutting down...
I20260812 02:43:27.860662 25587 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Refusing update from remote peer 2dce354ed6604f9dba212f727ac21f28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 02:43:27.861984 25651 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.252.keytab.
Entry for principal oryx/127.17.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.252.keytab.
I20260812 02:43:27.866842 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.252:40163
--webserver_interface=127.17.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.252.keytab
--principal=oryx/127.17.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:34567,127.17.196.253:46567,127.17.196.252:40163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:27.867493 25004 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 02:43:27.876549 25538 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:27.877123 25538 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 02:43:27.877091 25004 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 -> Peer b1f1396065e740869a40a86ff99bf02f (127.17.196.253:46567): Couldn't send request to peer b1f1396065e740869a40a86ff99bf02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 02:43:27.877341 25538 tablet_replica.cc:330] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: stopping tablet replica
I20260812 02:43:27.886358 25538 master.cc:552] Master@127.17.196.253:46567 shutdown complete.
I20260812 02:43:27.886549 25538 master_runner.cc:305] Clearing existing system tablet
I20260812 02:43:27.887049 25538 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:27.887154 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:27.890304 25538 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 02:43:27.892149 25657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:27.892546 25538 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:27.892834 25538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 02:43:27.892937 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
uuid: "b1f1396065e740869a40a86ff99bf02f"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "33ff4389ba7e55ba1acc4935e1682c69"
server_key_iv: "756a37d4777ca858127561a3975faa61"
server_key_version: "encryptionkey@0"
I20260812 02:43:27.898345 25538 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:27.906515 25538 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 02:43:27.906751 25538 log.cc:1196] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 02:43:27.907261 25538 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Deleting consensus metadata
I20260812 02:43:27.907555 25538 master_runner.cc:315] Copying system tablet from 127.17.196.254:34567
I20260812 02:43:27.910367 25538 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: tablet copy: Beginning tablet copy session from remote peer at address 127.17.196.254:34567
I20260812 02:43:27.934069 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.911615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48224 (local address 127.17.196.254:34567)
0812 02:43:27.911815 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:27.911824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:27.911848 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:43:27.912103 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:27.912109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:27.912194 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:27.912307 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:27.913401 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.914566 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.916540 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.917209 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.917318 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.920044 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:27.920051 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:27.920056 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:27.920098 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:27.928669 (+ 8571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.930388 (+ 1719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.930394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.930398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.930587 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:27.932116 (+ 1529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:27.932122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:27.932126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:27.932528 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:27.932683 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:27.933323 (+ 640us) server_negotiation.cc:299] Negotiation successful
0812 02:43:27.933620 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:43:27.934912 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:27.911209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:48224)
0812 02:43:27.911736 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:27.911763 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:27.911919 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:27.912367 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:27.912373 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:27.912957 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:27.913216 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.913229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.914787 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.914792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.916335 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:27.916354 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:27.917655 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:27.917660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:27.917847 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:27.918686 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:27.918710 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:27.919774 (+ 1064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:27.929599 (+ 9825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.929613 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.929619 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.930235 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.931728 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:27.931734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:27.931737 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:27.931941 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:27.932817 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:27.932822 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:27.933038 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 02:43:27.933210 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 02:43:27.933702 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":477,"threads_started":1}
I20260812 02:43:27.935952 25046 tablet_copy_service.cc:140] P 2dce354ed6604f9dba212f727ac21f28: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b1f1396065e740869a40a86ff99bf02f ({username='oryx', principal='oryx/127.17.196.253@KRBTEST.COM'} at 127.0.0.1:48224)
I20260812 02:43:27.936272 25046 tablet_copy_service.cc:161] P 2dce354ed6604f9dba212f727ac21f28: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b1f1396065e740869a40a86ff99bf02f at {username='oryx', principal='oryx/127.17.196.253@KRBTEST.COM'} at 127.0.0.1:48224: session id = b1f1396065e740869a40a86ff99bf02f-00000000000000000000000000000000
I20260812 02:43:27.940991 25046 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 02:43:27.943631 25538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 02:43:27.951468 25538 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: tablet copy: Starting download of 0 data blocks...
I20260812 02:43:27.952278 25538 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: tablet copy: Starting download of 1 WAL segments...
I20260812 02:43:27.955489 25538 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 02:43:27.960593 25046 tablet_copy_service.cc:342] P 2dce354ed6604f9dba212f727ac21f28: Request end of tablet copy session b1f1396065e740869a40a86ff99bf02f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.196.253@KRBTEST.COM'} at 127.0.0.1:48224
I20260812 02:43:27.960950 25046 tablet_copy_service.cc:433] P 2dce354ed6604f9dba212f727ac21f28: ending tablet copy session b1f1396065e740869a40a86ff99bf02f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b1f1396065e740869a40a86ff99bf02f
I20260812 02:43:27.974413 25538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:27.977792 25666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:27.978070 25667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:27.985507 25669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:27.987078 25538 server_base.cc:1034] running on GCE node
I20260812 02:43:27.987524 25538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:27.988394 25538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:27.989709 25538 hybrid_clock.cc:648] HybridClock initialized: now 1786502607989635 us; error 42 us; skew 500 ppm
I20260812 02:43:27.989956 25538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:27 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502607, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:27.992233 25538 init.cc:377] Logged in from keytab as oryx/127.17.196.253@KRBTEST.COM (short username oryx)
I20260812 02:43:27.993846 25538 webserver.cc:460] Webserver started at http://127.17.196.253:36743/ using document root <none> and password file <none>
I20260812 02:43:27.994251 25538 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:27.994341 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:27.998234 25538 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 02:43:28.000276 25679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:28.000777 25538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260812 02:43:28.000983 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
uuid: "b1f1396065e740869a40a86ff99bf02f"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "33ff4389ba7e55ba1acc4935e1682c69"
server_key_iv: "756a37d4777ca858127561a3975faa61"
server_key_version: "encryptionkey@0"
I20260812 02:43:28.001232 25538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:28.025467 25538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:28.030079 25538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:28.070572 25538 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.253:46567
I20260812 02:43:28.070601 25734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.253:46567 every 8 connection(s)
I20260812 02:43:28.071843 25538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
I20260812 02:43:28.078004 25735 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:43:28.082707 25735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Bootstrap starting.
I20260812 02:43:28.102257 25735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:43:28.102983 25735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Bootstrap complete.
I20260812 02:43:28.104446 25735 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } }
I20260812 02:43:28.104797 25735 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1f1396065e740869a40a86ff99bf02f, State: Initialized, Role: LEARNER
I20260812 02:43:28.105089 25735 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } }
I20260812 02:43:28.105969 25738 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } }
I20260812 02:43:28.106331 25735 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:28.106388 25738 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: This master's current role is: LEARNER
I20260812 02:43:28.113914 25751 catalog_manager.cc:1259] Loaded cluster ID: 84bd3a9d86e94265ae1b366d3829c551
I20260812 02:43:28.115661 25751 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: loading cluster ID for follower catalog manager: success
I20260812 02:43:28.119240 25751 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: acquiring CA information for follower catalog manager: success
I20260812 02:43:28.121380 25751 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:43:28.148602 25532 heartbeater.cc:503] Master 127.17.196.254:34567 was elected leader, sending a full tablet report...
W20260812 02:43:28.392047 25655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:28.392524 25655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:28.392804 25655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:28.404371 25655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:28.404582 25655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:28.404704 25655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:28.404825 25655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:28.404922 25655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:28.418900 25655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:34567,127.17.196.253:46567,127.17.196.252:40163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.252.keytab
--principal=oryx/127.17.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.252:40163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:28.420591 25655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:28.422858 25655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:43:28.433008 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.423485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.0.0.1:42740)
0812 02:43:28.424268 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.424296 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.424482 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.425405 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.425408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.425433 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:28.425732 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.425745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.427985 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.427989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.429381 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.429397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.431690 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.431698 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.431954 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.432119 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.432266 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.432474 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":428,"threads_started":1}
I20260812 02:43:28.433712 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.423855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.17.196.253:46567)
0812 02:43:28.424865 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.424872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.424929 (+ 57us) server_negotiation.cc:407] Connection header received
0812 02:43:28.425014 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.425019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.425186 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.425350 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:28.425905 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.427756 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.429681 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.431346 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.431749 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.432032 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.432347 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.432691 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":544,"threads_started":1}
I20260812 02:43:28.435662 25708 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260812 02:43:28.436396 25738 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dce354ed6604f9dba212f727ac21f28. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } }
I20260812 02:43:28.436692 25738 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: This master's current role is: LEARNER
W20260812 02:43:28.443851 25760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:28.445034 25763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:28.445034 25761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:28.445972 25655 server_base.cc:1034] running on GCE node
I20260812 02:43:28.447119 25655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:28.449105 25655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:28.450443 25655 hybrid_clock.cc:648] HybridClock initialized: now 1786502608450272 us; error 106 us; skew 500 ppm
I20260812 02:43:28.450924 25655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:28 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502608, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:28.456362 25655 init.cc:377] Logged in from keytab as oryx/127.17.196.252@KRBTEST.COM (short username oryx)
I20260812 02:43:28.459342 25655 webserver.cc:460] Webserver started at http://127.17.196.252:44789/ using document root <none> and password file <none>
I20260812 02:43:28.460235 25655 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:28.460407 25655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:28.460793 25655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:28.465312 25655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data/instance:
uuid: "60c87d55fe1943f99e0a51376de660c9"
format_stamp: "Formatted at 2026-08-12 02:43:28 on dist-test-slave-ss4n"
server_key: "b28c2386d95f7099cc4b52a250663b60"
server_key_iv: "90f16dd9289e01c7e289b5b6b4b5c9ae"
server_key_version: "encryptionkey@0"
I20260812 02:43:28.466470 25655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal/instance:
uuid: "60c87d55fe1943f99e0a51376de660c9"
format_stamp: "Formatted at 2026-08-12 02:43:28 on dist-test-slave-ss4n"
server_key: "b28c2386d95f7099cc4b52a250663b60"
server_key_iv: "90f16dd9289e01c7e289b5b6b4b5c9ae"
server_key_version: "encryptionkey@0"
I20260812 02:43:28.474030 25655 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.003s
I20260812 02:43:28.478814 25771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:28.480752 25655 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:43:28.481073 25655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
uuid: "60c87d55fe1943f99e0a51376de660c9"
format_stamp: "Formatted at 2026-08-12 02:43:28 on dist-test-slave-ss4n"
server_key: "b28c2386d95f7099cc4b52a250663b60"
server_key_iv: "90f16dd9289e01c7e289b5b6b4b5c9ae"
server_key_version: "encryptionkey@0"
I20260812 02:43:28.481415 25655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:28.507720 25655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:28.514319 25655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:28.514828 25655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:28.550704 25655 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.252:40163
I20260812 02:43:28.550812 25834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.252:40163 every 8 connection(s)
I20260812 02:43:28.552616 25655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
I20260812 02:43:28.556016 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 25655
I20260812 02:43:28.559476 25835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:28.569993 25835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:43:28 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.252@KRBTEST.COM
Aug 12 02:43:28 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502608, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.252@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:28.590502 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.558920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.17.196.252:40163)
0812 02:43:28.560926 (+ 2006us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.560980 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.561077 (+ 97us) server_negotiation.cc:407] Connection header received
0812 02:43:28.563839 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.563867 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.564327 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.564853 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.567100 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.568740 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.570515 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.571392 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.571612 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.578928 (+ 7316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.578939 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.578955 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.579026 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.583730 (+ 4704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.584964 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.584972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.584980 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.585150 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.585780 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.585784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.585787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.586735 (+ 948us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.587133 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.587921 (+ 788us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.588476 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1657,"thread_start_us":741,"threads_started":1}
I20260812 02:43:28.594529 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.573313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:48226)
0812 02:43:28.574711 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.574770 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.574991 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.575626 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.575632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.576562 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:28.577132 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.577144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.578820 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.578828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.580470 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.580486 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.581974 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.581981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.582199 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.582942 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:28.582961 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:28.586227 (+ 3266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:28.591396 (+ 5169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.591407 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.591413 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.592090 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.592524 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.592528 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.592530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.592704 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.593366 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:28.593372 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:28.593670 (+ 298us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.593903 (+ 233us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.594226 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"thread_start_us":716,"threads_started":1}
I20260812 02:43:28.594676 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.573241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.17.196.254:34567)
0812 02:43:28.574081 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.574091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.574910 (+ 819us) server_negotiation.cc:407] Connection header received
0812 02:43:28.575177 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.575185 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.575313 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.575471 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.577345 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.578537 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.580668 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.581669 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.581851 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.586457 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.586463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.586467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.586511 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.591222 (+ 4711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.592208 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.592213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.592216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.592383 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.592829 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.592833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.592837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.593215 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.593357 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.594028 (+ 671us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.594213 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":485,"threads_started":1}
I20260812 02:43:28.595978 25835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:43:28 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502608, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.252@KRBTEST.COM for oryx/127.17.196.253@KRBTEST.COM
I20260812 02:43:28.614260 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.596919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.17.196.253:46567)
0812 02:43:28.597096 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.597100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.597127 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:43:28.597329 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.597334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.597408 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.597600 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.598608 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.599575 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.601224 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.602236 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.602353 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.606314 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.606320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.606324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.606388 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.610362 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.611735 (+ 1373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.611741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.611744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.611985 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.612597 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.612601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.612603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.612976 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.613101 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.613731 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.613961 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 02:43:28.614291 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.596774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.0.0.1:42756)
0812 02:43:28.597053 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.597070 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.597154 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.597602 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.597605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.598221 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:28.598409 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.598417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.599729 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.599733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.601006 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.601019 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.602471 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.602478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.602682 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.603494 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:28.603515 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:28.605983 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:28.610664 (+ 4681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.610668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.610679 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.611345 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.612193 (+ 848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.612198 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.612200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.612469 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.613187 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:28.613192 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:28.613370 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.613643 (+ 273us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.613969 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 02:43:28.615841 25835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:43:28 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502608, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.252@KRBTEST.COM for oryx/127.17.196.252@KRBTEST.COM
I20260812 02:43:28.632100 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.616955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47480 (local address 127.17.196.252:40163)
0812 02:43:28.617308 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.617315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.617421 (+ 106us) server_negotiation.cc:407] Connection header received
0812 02:43:28.617749 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.617754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.617822 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.617930 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.618960 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.619832 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.621256 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.621838 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.621988 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.625559 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.625564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.625567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.625596 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.628975 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.629792 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.629799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.629803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.629930 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.630406 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.630413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.630416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.630702 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.630864 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.631219 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.631534 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260812 02:43:28.632355 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.617166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.0.0.1:47480)
0812 02:43:28.617307 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.617329 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.617616 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.617946 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.617950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.618569 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:28.618810 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.618819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.619980 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.619988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.621104 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.621117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.622054 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.622061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.622267 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.622990 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:28.623012 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:28.625342 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:28.629144 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.629150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.629155 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.629676 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.630071 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.630075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.630077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.630293 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.630826 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:28.630830 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:28.630994 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.631141 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.631691 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":209}
I20260812 02:43:28.638844 25835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Bootstrap starting.
I20260812 02:43:28.642900 25835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:28.644884 25835 log.cc:824] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:28.648656 25835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: No bootstrap required, opened a new log
I20260812 02:43:28.659055 25835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } }
I20260812 02:43:28.659584 25835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:28.659886 25835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c87d55fe1943f99e0a51376de660c9, State: Initialized, Role: FOLLOWER
I20260812 02:43:28.660626 25835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } }
I20260812 02:43:28.662062 25842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } } }
I20260812 02:43:28.662688 25842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:43:28.663256 25835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 02:43:28.678381 25854 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:43:28.678681 25854 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:43:28.683408 25655 master_runner.cc:428] Detected that this master 60c87d55fe1943f99e0a51376de660c9 is joining an existing cluster
I20260812 02:43:28.683764 25655 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.196.252:40163
I20260812 02:43:28.698565 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 25090
Aug 12 02:43:28 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.193@KRBTEST.COM
I20260812 02:43:28.727598 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.700665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.17.196.193:34391)
0812 02:43:28.701838 (+ 1173us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.701849 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.701885 (+ 36us) server_negotiation.cc:407] Connection header received
0812 02:43:28.701990 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.702002 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.702433 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.702693 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.704973 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.707034 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.709292 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.710486 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.716737 (+ 6251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.719593 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.719613 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.719635 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.719679 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.724218 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.725014 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.725025 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.725028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.725188 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.725695 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.725701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.725704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.726078 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.726432 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.726794 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.727113 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":544,"threads_started":1}
I20260812 02:43:28.729281 25230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:28.742676 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.691754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.17.196.253:46567)
0812 02:43:28.692684 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.692691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.694972 (+ 2281us) server_negotiation.cc:407] Connection header received
0812 02:43:28.695207 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.695216 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.695331 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.695463 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.703679 (+ 8216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.704992 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.708288 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.709258 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.709363 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.714695 (+ 5332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.714703 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.714709 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.714773 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.720063 (+ 5290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.734732 (+ 14669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.734741 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.734744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.735066 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.735643 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.735648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.735651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.736106 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.736289 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.741684 (+ 5395us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.742069 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747}
I20260812 02:43:28.744304 25860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.691589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.0.0.1:42768)
0812 02:43:28.694773 (+ 3184us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.694808 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.695032 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.695719 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.695723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.698554 (+ 2831us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:28.702952 (+ 4398us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x61200002d828. stack: 00007f69e4ad966e 00007f69e4ad9430 00007f69e3b605ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f69e4acf0fd 00007f69e4acf977 00007f69f8e7da7d 00007f69f8e7da0e 00007f69e5c087b7 00007f69eb4dd03a 00007f69eb564dca 00007f69eb562f68 00007f69eb59682f 00007f69eb59658d
0812 02:43:28.703450 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.703470 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.705366 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.705376 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.708107 (+ 2731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.708123 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.709828 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.709834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.710162 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.712237 (+ 2075us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:28.712261 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:28.714327 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:28.720240 (+ 5913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.720246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.720250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.734415 (+ 14165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.735250 (+ 835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.735256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.735260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.735485 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.736366 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:28.736374 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:28.738503 (+ 2129us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.743003 (+ 4500us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.743459 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2945,"mutex_wait_us":38,"spinlock_wait_cycles":5016448,"thread_start_us":2468,"threads_started":1}
I20260812 02:43:28.746603 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.691523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.17.196.254:34567)
0812 02:43:28.692468 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.692475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.712057 (+ 19582us) server_negotiation.cc:407] Connection header received
0812 02:43:28.712482 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.712488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.712638 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.712729 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.715494 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.716759 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.719458 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.720214 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.720319 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.735747 (+ 15428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.735756 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.735763 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.735822 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.741250 (+ 5428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.742527 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.742535 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.742540 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.742699 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.743637 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.743642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.743646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.744095 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.745542 (+ 1447us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.745692 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.746301 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787}
I20260812 02:43:28.747825 25862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.692703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:48242)
0812 02:43:28.711944 (+ 19241us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.711965 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.712234 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.713042 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.713046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.714505 (+ 1459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:28.715275 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.715293 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.717025 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.717032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.719271 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.719286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.720641 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.720649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.720928 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.721710 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:28.721742 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:28.735391 (+ 13649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:28.741376 (+ 5985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.741380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.741384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.742350 (+ 966us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.743177 (+ 827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.743181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.743184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.743486 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.744246 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:28.744251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:28.744953 (+ 702us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.745140 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.746070 (+ 930us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17977,"mutex_wait_us":20,"spinlock_wait_cycles":7168,"thread_start_us":6160,"threads_started":1}
I20260812 02:43:28.747867 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.691258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47482 (local address 127.17.196.252:40163)
0812 02:43:28.692263 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.692271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.698959 (+ 6688us) server_negotiation.cc:407] Connection header received
0812 02:43:28.701829 (+ 2870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.701836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.701946 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.702044 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.705564 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.707276 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.710445 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.711580 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.711818 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.735660 (+ 23842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.735667 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.735672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.735712 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.741200 (+ 5488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.742485 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.742493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.742498 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.742661 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.743100 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.743107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.743111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.743832 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.744055 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.746181 (+ 2126us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.746424 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"spinlock_wait_cycles":2304}
I20260812 02:43:28.749085 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.691825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.0.0.1:47482)
0812 02:43:28.698820 (+ 6995us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.698846 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.701640 (+ 2794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.703737 (+ 2097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.703744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.704805 (+ 1061us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:28.705338 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.705355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.707801 (+ 2446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.707807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.710243 (+ 2436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.710259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.713290 (+ 3031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.713302 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.715607 (+ 2305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.720478 (+ 4871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:28.720504 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:28.735420 (+ 14916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:28.741373 (+ 5953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.741378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.741382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.742350 (+ 968us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.742804 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.742810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.742814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.742980 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.744131 (+ 1151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:28.744138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:28.744444 (+ 306us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.745732 (+ 1288us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.748367 (+ 2635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6564,"mutex_wait_us":44,"spinlock_wait_cycles":13312,"thread_start_us":5302,"threads_started":1}
W20260812 02:43:28.767441 25777 tablet.cc:2369] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 02:43:28 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502605, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.254@KRBTEST.COM for oryx/127.17.196.252@KRBTEST.COM
I20260812 02:43:28.778023 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.755839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.0.0.1:47498)
0812 02:43:28.756207 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.756227 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.756350 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.757290 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.757296 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.758003 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:28.758341 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.758357 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.760749 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.760755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.762931 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.762950 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.764404 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.764410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.764765 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.765633 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:28.765660 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:28.768741 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:28.774055 (+ 5314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.774063 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.774074 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.774940 (+ 866us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.775524 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.775530 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.775534 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.775812 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.776571 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:28.776577 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:28.776907 (+ 330us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.777086 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.777523 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260812 02:43:28.778227 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.756563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.17.196.252:40163)
0812 02:43:28.756782 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.756789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.756808 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:43:28.756928 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.756933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.757101 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.757249 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.758512 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.760378 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.763140 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.764186 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.764313 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.768991 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.769000 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.769006 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.769052 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.773826 (+ 4774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.775155 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.775162 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.775166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.775368 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.775985 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.775990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.775994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.776412 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.776573 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.777333 (+ 760us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.777674 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:43:28.780176 25024 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.196.252:40163: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "60c87d55fe1943f99e0a51376de660c9"
member_type: NON_VOTER
last_known_addr {
host: "127.17.196.252"
port: 40163
}
attrs {
promote: true
}
}
dest_uuid: "2dce354ed6604f9dba212f727ac21f28"
cas_config_opid_index: 5
I20260812 02:43:28.781320 25024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } }
I20260812 02:43:28.783473 25756 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 60c87d55fe1943f99e0a51376de660c9 (127.17.196.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.784863 25805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:28.787286 25708 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 LEARNER]: Refusing update from remote peer 2dce354ed6604f9dba212f727ac21f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 02:43:28.790764 25805 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Refusing update from remote peer 2dce354ed6604f9dba212f727ac21f28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 02:43:28.793025 25866 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 02:43:28.797091 25708 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 60c87d55fe1943f99e0a51376de660c9 (127.17.196.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.804299 25756 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.196.252:40163
I20260812 02:43:28.804665 25867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:28.805303 25655 master.cc:537] Master@127.17.196.252:40163 shutting down...
I20260812 02:43:28.810719 25738 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.811064 25738 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: This master's current role is: LEARNER
I20260812 02:43:28.813367 25842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:28.814940 25842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } }
I20260812 02:43:28.816658 25842 leader_election.cc:288] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2dce354ed6604f9dba212f727ac21f28 (127.17.196.254:34567), b1f1396065e740869a40a86ff99bf02f (127.17.196.253:46567)
I20260812 02:43:28.817765 25708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60c87d55fe1943f99e0a51376de660c9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f1396065e740869a40a86ff99bf02f" is_pre_election: true
I20260812 02:43:28.806039 25866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: NON_VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: true } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.817407 25035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60c87d55fe1943f99e0a51376de660c9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dce354ed6604f9dba212f727ac21f28" is_pre_election: true
I20260812 02:43:28.822757 25774 leader_election.cc:302] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60c87d55fe1943f99e0a51376de660c9; no voters: 2dce354ed6604f9dba212f727ac21f28, b1f1396065e740869a40a86ff99bf02f
I20260812 02:43:28.823309 25842 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 02:43:28.824505 25866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:28.827212 25756 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: attempting to promote NON_VOTER b1f1396065e740869a40a86ff99bf02f to VOTER
I20260812 02:43:28.827548 25655 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:28.828063 25655 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:28.828359 25655 tablet_replica.cc:330] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: stopping tablet replica
I20260812 02:43:28.828217 25756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } }
I20260812 02:43:28.833261 25708 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 LEARNER]: Refusing update from remote peer 2dce354ed6604f9dba212f727ac21f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260812 02:43:28.834834 25007 connection.cc:535] client connection to 127.17.196.252:40163 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:43:28.837702 25867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 02:43:28.839381 25655 master.cc:552] Master@127.17.196.252:40163 shutdown complete.
I20260812 02:43:28.843075 25871 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b1f1396065e740869a40a86ff99bf02f (127.17.196.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.843329 25708 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b1f1396065e740869a40a86ff99bf02f (127.17.196.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.846851 25655 master_runner.cc:305] Clearing existing system tablet
I20260812 02:43:28.847500 25655 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:28.847711 25655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:28.848251 25871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.848721 25871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:28.849165 25868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.849604 25868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:28.852458 25738 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:28.852793 25738 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:43:28.855407 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.854808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.0.0.1:47508)
0812 02:43:28.855004 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.855194 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 02:43:28.855938 25007 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 -> Peer 60c87d55fe1943f99e0a51376de660c9 (127.17.196.252:40163): Couldn't send request to peer 60c87d55fe1943f99e0a51376de660c9. Status: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 02:43:28.866384 25655 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.002s sys 0.010s
I20260812 02:43:28.868685 25874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:28.869195 25655 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:28.869546 25655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 02:43:28.869680 25655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
uuid: "60c87d55fe1943f99e0a51376de660c9"
format_stamp: "Formatted at 2026-08-12 02:43:28 on dist-test-slave-ss4n"
server_key: "b28c2386d95f7099cc4b52a250663b60"
server_key_iv: "90f16dd9289e01c7e289b5b6b4b5c9ae"
server_key_version: "encryptionkey@0"
I20260812 02:43:28.872174 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 25090
I20260812 02:43:28.875833 25655 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:28.885289 25655 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 02:43:28.885576 25655 log.cc:1196] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 02:43:28.886317 25655 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Deleting consensus metadata
I20260812 02:43:28.886615 25655 master_runner.cc:315] Copying system tablet from 127.17.196.254:34567
I20260812 02:43:28.891337 25655 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: tablet copy: Beginning tablet copy session from remote peer at address 127.17.196.254:34567
I20260812 02:43:28.898300 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:34391
--local_ip_for_outbound_sockets=127.17.196.193
--tserver_master_addrs=127.17.196.254:34567
--webserver_port=40129
--webserver_interface=127.17.196.193
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab
--principal=oryx/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567,127.17.196.252:40163 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:43:28.911733 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.892272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:48250)
0812 02:43:28.893930 (+ 1658us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:28.893952 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:28.894193 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:28.894707 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:28.894713 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:28.895314 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:28.895604 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.895618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.897331 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.897341 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.898928 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:28.898945 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.900223 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.900229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:28.900425 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.901287 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:28.901311 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:28.902606 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:28.907914 (+ 5308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.907921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.907926 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.908574 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.909216 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:28.909223 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:28.909228 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:28.909514 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:28.910393 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:28.910398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:28.910652 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 02:43:28.910820 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 02:43:28.911238 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1502,"thread_start_us":1373,"threads_started":1}
I20260812 02:43:28.912256 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:28.892346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.17.196.254:34567)
0812 02:43:28.892586 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:28.892593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:28.894055 (+ 1462us) server_negotiation.cc:407] Connection header received
0812 02:43:28.894404 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:28.894414 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:28.894522 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:28.894654 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:28.895811 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.897134 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.899118 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:28.899808 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:28.899909 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:28.902912 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:28.902924 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:28.902933 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:28.902998 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:28.907708 (+ 4710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.908797 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.908808 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.908813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.909064 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:28.909675 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:28.909684 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:28.909690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:28.910228 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:28.910416 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:28.911024 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 02:43:28.911377 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260812 02:43:28.912847 25046 tablet_copy_service.cc:140] P 2dce354ed6604f9dba212f727ac21f28: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 60c87d55fe1943f99e0a51376de660c9 ({username='oryx', principal='oryx/127.17.196.252@KRBTEST.COM'} at 127.0.0.1:48250)
I20260812 02:43:28.913043 25046 tablet_copy_service.cc:161] P 2dce354ed6604f9dba212f727ac21f28: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 60c87d55fe1943f99e0a51376de660c9 at {username='oryx', principal='oryx/127.17.196.252@KRBTEST.COM'} at 127.0.0.1:48250: session id = 60c87d55fe1943f99e0a51376de660c9-00000000000000000000000000000000
I20260812 02:43:28.916906 25046 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 02:43:28.919103 25655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 02:43:28.926870 25655 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: tablet copy: Starting download of 0 data blocks...
I20260812 02:43:28.927242 25655 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: tablet copy: Starting download of 1 WAL segments...
I20260812 02:43:28.930444 25655 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 02:43:28.935200 25046 tablet_copy_service.cc:342] P 2dce354ed6604f9dba212f727ac21f28: Request end of tablet copy session 60c87d55fe1943f99e0a51376de660c9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.196.252@KRBTEST.COM'} at 127.0.0.1:48250
I20260812 02:43:28.935359 25046 tablet_copy_service.cc:433] P 2dce354ed6604f9dba212f727ac21f28: ending tablet copy session 60c87d55fe1943f99e0a51376de660c9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 60c87d55fe1943f99e0a51376de660c9
I20260812 02:43:28.948179 25655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:28.951882 25884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:28.959988 25885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:28.965250 25655 server_base.cc:1034] running on GCE node
W20260812 02:43:28.965566 25887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:28.966123 25655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:28.966907 25655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:28.968281 25655 hybrid_clock.cc:648] HybridClock initialized: now 1786502608968210 us; error 48 us; skew 500 ppm
I20260812 02:43:28.968568 25655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:28 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502608, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:28.971345 25655 init.cc:377] Logged in from keytab as oryx/127.17.196.252@KRBTEST.COM (short username oryx)
I20260812 02:43:28.979285 25655 webserver.cc:460] Webserver started at http://127.17.196.252:33789/ using document root <none> and password file <none>
I20260812 02:43:28.979717 25655 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:28.979846 25655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:28.983618 25655 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:43:28.985997 25895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:28.986604 25655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 02:43:28.986753 25655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
uuid: "60c87d55fe1943f99e0a51376de660c9"
format_stamp: "Formatted at 2026-08-12 02:43:28 on dist-test-slave-ss4n"
server_key: "b28c2386d95f7099cc4b52a250663b60"
server_key_iv: "90f16dd9289e01c7e289b5b6b4b5c9ae"
server_key_version: "encryptionkey@0"
I20260812 02:43:28.986971 25655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:29.002002 25655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:29.007153 25655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:29.041674 25655 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.252:40163
I20260812 02:43:29.041736 25959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.252:40163 every 8 connection(s)
I20260812 02:43:29.042862 25655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
I20260812 02:43:29.048780 25960 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:43:29.053126 25960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Bootstrap starting.
I20260812 02:43:29.076030 25960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:43:29.076738 25960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: Bootstrap complete.
I20260812 02:43:29.078135 25960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } }
I20260812 02:43:29.078495 25960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 60c87d55fe1943f99e0a51376de660c9, State: Initialized, Role: LEARNER
I20260812 02:43:29.078815 25960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } }
I20260812 02:43:29.090253 25964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:29.090808 25964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: This master's current role is: LEARNER
I20260812 02:43:29.091053 25960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:29.100981 25976 catalog_manager.cc:1259] Loaded cluster ID: 84bd3a9d86e94265ae1b366d3829c551
I20260812 02:43:29.101143 25976 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: loading cluster ID for follower catalog manager: success
I20260812 02:43:29.104564 25976 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: acquiring CA information for follower catalog manager: success
I20260812 02:43:29.107218 25976 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:43:29.240011 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.222342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.0.0.1:47518)
0812 02:43:29.222738 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:29.222760 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:29.222938 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:29.232371 (+ 9433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:29.232378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:29.232422 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:29.232664 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:29.232681 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.234844 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.234851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:29.236893 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:29.236928 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.239019 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.239026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:29.239298 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.239353 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 02:43:29.239484 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 02:43:29.239660 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260812 02:43:29.240098 25977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.223274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.17.196.252:40163)
0812 02:43:29.231806 (+ 8532us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:29.231813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:29.231848 (+ 35us) server_negotiation.cc:407] Connection header received
0812 02:43:29.231952 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:29.231959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:29.232077 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:29.232272 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:29.232872 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.234637 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.237190 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.238763 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.239009 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.239234 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:29.239541 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 02:43:29.239706 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8373,"thread_start_us":8167,"threads_started":1}
I20260812 02:43:29.241335 25930 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 02:43:29.242240 25964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dce354ed6604f9dba212f727ac21f28. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:29.242594 25964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: This master's current role is: LEARNER
W20260812 02:43:29.428422 25883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:29.428860 25883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:29.429113 25883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:29.438982 25883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:29.439208 25883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:29.439414 25883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:29.453908 25883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab
--principal=oryx/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:34391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=40129
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567,127.17.196.252:40163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:29.455639 25883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:29.457545 25883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:29.478192 25984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:29.478645 25986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:29.479414 25983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:29.479990 25883 server_base.cc:1034] running on GCE node
I20260812 02:43:29.480644 25883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:29.481729 25883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:29.482986 25883 hybrid_clock.cc:648] HybridClock initialized: now 1786502609482904 us; error 73 us; skew 500 ppm
I20260812 02:43:29.483398 25883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:29 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502609, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:29.488649 25883 init.cc:377] Logged in from keytab as oryx/127.17.196.193@KRBTEST.COM (short username oryx)
I20260812 02:43:29.490437 25883 webserver.cc:460] Webserver started at http://127.17.196.193:40129/ using document root <none> and password file <none>
I20260812 02:43:29.491466 25883 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:29.491678 25883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:29.498787 25883 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:43:29.502771 25993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:29.504382 25883 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:43:29.504578 25883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "f8a58be1415e4d268335e44dec2c1baf"
format_stamp: "Formatted at 2026-08-12 02:43:25 on dist-test-slave-ss4n"
server_key: "67432f6b7a06fdb3ec4b80f2b346a1d2"
server_key_iv: "15453256bc9037e3639992254e16ead5"
server_key_version: "encryptionkey@0"
I20260812 02:43:29.506139 25883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:29.520562 25883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:29.526922 25883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:29.527391 25883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:29.529210 25883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:29.531072 25883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:29.531260 25883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:29.531402 25883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:29.531576 25883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:29.603790 25883 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:34391
I20260812 02:43:29.604076 26125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:34391 every 8 connection(s)
I20260812 02:43:29.605664 25883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:29.606681 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 25883
I20260812 02:43:29.607256 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 25242
Aug 12 02:43:29 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.194@KRBTEST.COM
Aug 12 02:43:29 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502609, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.252@KRBTEST.COM
I20260812 02:43:29.662017 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.608881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.17.196.194:42369)
0812 02:43:29.620759 (+ 11878us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:29.620772 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:29.620811 (+ 39us) server_negotiation.cc:407] Connection header received
0812 02:43:29.620904 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:29.620918 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:29.621669 (+ 751us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:29.621927 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:29.625842 (+ 3915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.628574 (+ 2732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.631912 (+ 3338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.633436 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.633595 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.647014 (+ 13419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:29.647047 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:29.647076 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:29.647147 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:29.654588 (+ 7441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:29.657467 (+ 2879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:29.657501 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:29.657508 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:29.657745 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:29.658339 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:29.658348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:29.658354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:29.659002 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:29.659376 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:29.660856 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0812 02:43:29.661266 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11634,"thread_start_us":11476,"threads_started":1}
I20260812 02:43:29.670863 25376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:29.676645 25867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: NON_VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: true } } }
I20260812 02:43:29.677258 25867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
Aug 12 02:43:29 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502609, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:29.686712 26140 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: attempting to promote NON_VOTER 60c87d55fe1943f99e0a51376de660c9 to VOTER
I20260812 02:43:29.687724 26140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } }
I20260812 02:43:29.688872 25977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.610966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:59385 (local address 127.17.196.252:40163)
0812 02:43:29.611353 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:29.611363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:29.623090 (+ 11727us) server_negotiation.cc:407] Connection header received
0812 02:43:29.627059 (+ 3969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:29.627068 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:29.627194 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:29.627334 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:29.640556 (+ 13222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.642413 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.647234 (+ 4821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.648404 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.648548 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.665797 (+ 17249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:29.665807 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:29.665814 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:29.665890 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:29.671921 (+ 6031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:29.678158 (+ 6237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:29.678168 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:29.678175 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:29.678603 (+ 428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:29.681611 (+ 3008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:29.681618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:29.681622 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:29.682117 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:29.682323 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:29.687475 (+ 5152us) server_negotiation.cc:299] Negotiation successful
0812 02:43:29.687956 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260812 02:43:29.691272 25708 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Refusing update from remote peer 2dce354ed6604f9dba212f727ac21f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 02:43:29.692235 26139 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 02:43:29.696631 25867 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 60c87d55fe1943f99e0a51376de660c9 (127.17.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:29.692165 25930 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 LEARNER]: Refusing update from remote peer 2dce354ed6604f9dba212f727ac21f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 02:43:29.697278 25708 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 60c87d55fe1943f99e0a51376de660c9 (127.17.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:29.702656 26140 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 02:43:29.703789 25867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:29.704391 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.616010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.17.196.193:59385)
0812 02:43:29.622811 (+ 6801us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:29.622935 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:29.626642 (+ 3707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:29.637396 (+ 10754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:29.637415 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:29.638751 (+ 1336us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:29.640210 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:29.640248 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.642860 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.642870 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:29.646941 (+ 4071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:29.646969 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.649267 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.649289 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:29.649843 (+ 554us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.651015 (+ 1172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:29.651058 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:29.662810 (+ 11752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:29.672581 (+ 9771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:29.672597 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:29.672620 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:29.673690 (+ 1070us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:29.679900 (+ 6210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:29.679906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:29.679910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:29.680404 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:29.682433 (+ 2029us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:29.682442 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:29.683107 (+ 665us) client_negotiation.cc:769] Sending connection context
0812 02:43:29.687283 (+ 4176us) client_negotiation.cc:240] Negotiation successful
0812 02:43:29.689540 (+ 2257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6205,"thread_start_us":9294,"threads_started":1}
I20260812 02:43:29.704433 25867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:29.704859 25867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:29.705349 25867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:29.706691 26126 heartbeater.cc:347] Connected to a master server at 127.17.196.252:40163
I20260812 02:43:29.707103 26126 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:29.708207 26126 heartbeater.cc:511] Master 127.17.196.252:40163 requested a full tablet report, sending...
I20260812 02:43:29.708741 26144 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:29.711098 26144 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:43:29.714282 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.621240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:54371 (local address 127.17.196.254:34567)
0812 02:43:29.636324 (+ 15084us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:29.636333 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:29.636359 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:43:29.636435 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:29.636454 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:29.636551 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:29.636686 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:29.656276 (+ 19590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.657779 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.667015 (+ 9236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.678266 (+ 11251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.678464 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.689662 (+ 11198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:29.689675 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:29.689683 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:29.689742 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:29.706342 (+ 16600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:29.711486 (+ 5144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:29.711493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:29.711497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:29.711690 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:29.712276 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:29.712282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:29.712286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:29.712641 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:29.712787 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:29.713348 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 02:43:29.713661 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14915,"thread_start_us":15612,"threads_started":1}
I20260812 02:43:29.714498 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.621322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.193:54371)
0812 02:43:29.632079 (+ 10757us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:29.632107 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:29.632308 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:29.648768 (+ 16460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:29.648774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:29.650265 (+ 1491us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:29.656005 (+ 5740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:29.656030 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.658023 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.658032 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:29.664016 (+ 5984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:29.664048 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.679060 (+ 15012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.679070 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:29.679422 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.680734 (+ 1312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:29.680758 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:29.689164 (+ 8406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:29.709226 (+ 20062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:29.709238 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:29.709246 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:29.710299 (+ 1053us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:29.711859 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:29.711865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:29.711868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:29.712124 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:29.712795 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:29.712801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:29.712997 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 02:43:29.713188 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 02:43:29.713629 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10506,"mutex_wait_us":4,"thread_start_us":11065,"threads_started":1}
I20260812 02:43:29.714856 25930 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 60c87d55fe1943f99e0a51376de660c9 (127.17.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:29.715974 26129 heartbeater.cc:347] Connected to a master server at 127.17.196.254:34567
I20260812 02:43:29.716255 26129 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:29.717279 26129 heartbeater.cc:511] Master 127.17.196.254:34567 requested a full tablet report, sending...
Aug 12 02:43:29 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502609, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.253@KRBTEST.COM
I20260812 02:43:29.719063 25024 ts_manager.cc:194] Re-registered known tserver with Master: f8a58be1415e4d268335e44dec2c1baf (127.17.196.193:34391)
I20260812 02:43:29.720206 25024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:54371
I20260812 02:43:29.724395 25964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:29.724740 25964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:43:29.726444 25918 ts_manager.cc:194] Registered new tserver with Master: f8a58be1415e4d268335e44dec2c1baf (127.17.196.193:34391)
I20260812 02:43:29.733208 26137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.634828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.17.196.193:51611)
0812 02:43:29.665299 (+ 30471us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:29.665323 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:29.665468 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:29.673863 (+ 8395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:29.673871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:29.674897 (+ 1026us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:29.675740 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:29.675760 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.684836 (+ 9076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.684844 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:29.687009 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:29.687028 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.713186 (+ 26158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.713194 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:29.713449 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.714403 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:29.714451 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:29.720055 (+ 5604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:29.729425 (+ 9370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:29.729432 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:29.729436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:29.730218 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:29.730778 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:29.730786 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:29.730792 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:29.731043 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:29.731736 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:29.731745 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:29.731988 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 02:43:29.732170 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 02:43:29.732591 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30279,"mutex_wait_us":28,"thread_start_us":30555,"threads_started":1}
I20260812 02:43:29.733266 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:29.635614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:51611 (local address 127.17.196.253:46567)
0812 02:43:29.661840 (+ 26226us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:29.661849 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:29.665553 (+ 3704us) server_negotiation.cc:407] Connection header received
0812 02:43:29.669080 (+ 3527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:29.669126 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:29.672185 (+ 3059us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:29.672298 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:29.675993 (+ 3695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.677781 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.710906 (+ 33125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:29.712383 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:29.717593 (+ 5210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:29.720348 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:29.720358 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:29.720364 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:29.720412 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:29.729184 (+ 8772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:29.730414 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:29.730420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:29.730422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:29.730612 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:29.731201 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:29.731207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:29.731209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:29.731527 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:29.731722 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:29.732303 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 02:43:29.732587 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26004,"thread_start_us":25864,"threads_started":1}
I20260812 02:43:29.734551 26133 heartbeater.cc:347] Connected to a master server at 127.17.196.253:46567
I20260812 02:43:29.734760 26133 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:29.735486 26133 heartbeater.cc:511] Master 127.17.196.253:46567 requested a full tablet report, sending...
I20260812 02:43:29.737684 25696 ts_manager.cc:194] Registered new tserver with Master: f8a58be1415e4d268335e44dec2c1baf (127.17.196.193:34391)
I20260812 02:43:29.785797 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 25242
I20260812 02:43:29.809577 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:42369
--local_ip_for_outbound_sockets=127.17.196.194
--tserver_master_addrs=127.17.196.254:34567
--webserver_port=34895
--webserver_interface=127.17.196.194
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab
--principal=oryx/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567,127.17.196.252:40163 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:30.185912 26146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:30.186290 26146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:30.186493 26146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:30.194396 26146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:30.194525 26146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:30.194710 26146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:30.207520 26146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab
--principal=oryx/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:42369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=34895
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567,127.17.196.252:40163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:30.209219 26146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:30.211076 26146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:43:30.216900 26150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dce354ed6604f9dba212f727ac21f28" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:30.217258 26150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28 [sys.catalog]: This master's current role is: LEADER
W20260812 02:43:30.228358 26155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:30.228384 26156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:30.228585 26159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:30.229317 26146 server_base.cc:1034] running on GCE node
I20260812 02:43:30.230051 26146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:30.231143 26146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:30.232388 26146 hybrid_clock.cc:648] HybridClock initialized: now 1786502610232343 us; error 41 us; skew 500 ppm
I20260812 02:43:30.232790 26146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:30 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502610, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:30.236250 26146 init.cc:377] Logged in from keytab as oryx/127.17.196.194@KRBTEST.COM (short username oryx)
I20260812 02:43:30.238128 26146 webserver.cc:460] Webserver started at http://127.17.196.194:34895/ using document root <none> and password file <none>
I20260812 02:43:30.238831 26146 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:30.238948 26146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:30.244865 26146 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:43:30.248378 26167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:30.250198 26146 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:43:30.250388 26146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "4af9696aa3b441cabaf65f148ff973f2"
format_stamp: "Formatted at 2026-08-12 02:43:26 on dist-test-slave-ss4n"
server_key: "8d056b20375dbaee3e3f616ea135ab1d"
server_key_iv: "d43fc63da87be53a8c981b064ce492a3"
server_key_version: "encryptionkey@0"
I20260812 02:43:30.251502 26146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:30.276615 26146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:30.282014 26146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:30.282326 26146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:30.283396 26146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:30.284878 26146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:30.284970 26146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:30.285046 26146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:30.285104 26146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:30.324560 26146 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:42369
I20260812 02:43:30.324582 26293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:42369 every 8 connection(s)
I20260812 02:43:30.326329 26146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:30.331218 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 26146
I20260812 02:43:30.331617 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 25388
Aug 12 02:43:30 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502610, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.252@KRBTEST.COM
Aug 12 02:43:30 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502610, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.253@KRBTEST.COM
Aug 12 02:43:30 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.195@KRBTEST.COM
Aug 12 02:43:30 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502610, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:30.381328 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:30.329871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:46171 (local address 127.17.196.253:46567)
0812 02:43:30.352151 (+ 22280us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:30.352159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:30.352183 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:43:30.352248 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:30.352252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:30.352339 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:30.352475 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:30.355434 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.358328 (+ 2894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.361913 (+ 3585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.362720 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.363490 (+ 770us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:30.370126 (+ 6636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:30.370134 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:30.370148 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:30.370207 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:30.376407 (+ 6200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:30.377701 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:30.377708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:30.377711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:30.377906 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:30.378560 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:30.378580 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:30.378584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:30.379121 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:30.379267 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:30.380455 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0812 02:43:30.380814 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22123,"thread_start_us":21937,"threads_started":1}
I20260812 02:43:30.381336 26299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:30.329584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:49307 (local address 127.17.196.252:40163)
0812 02:43:30.345022 (+ 15438us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:30.345033 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:30.353836 (+ 8803us) server_negotiation.cc:407] Connection header received
0812 02:43:30.354097 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:30.354103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:30.354184 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:30.354312 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:30.356872 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.358939 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.361928 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.362689 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.362812 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:30.369652 (+ 6840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:30.369658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:30.369661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:30.369703 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:30.375175 (+ 5472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:30.377037 (+ 1862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:30.377044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:30.377049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:30.377278 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:30.378029 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:30.378037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:30.378040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:30.378547 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:30.378728 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:30.380238 (+ 1510us) server_negotiation.cc:299] Negotiation successful
0812 02:43:30.380599 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15235,"thread_start_us":15448,"threads_started":1}
I20260812 02:43:30.383852 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:30.334515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.17.196.194:49307)
0812 02:43:30.353721 (+ 19206us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:30.353744 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:30.353939 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:30.354339 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:30.354343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:30.355668 (+ 1325us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:30.356617 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:30.356637 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.359158 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.359174 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:30.361647 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:30.361663 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.363139 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.363144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:30.363453 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:30.364483 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:30.364509 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:30.369316 (+ 4807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:30.375429 (+ 6113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:30.375496 (+ 67us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:30.375522 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:30.376793 (+ 1271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:30.377512 (+ 719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:30.377517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:30.377521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:30.377804 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:30.378836 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:30.378849 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:30.379350 (+ 501us) client_negotiation.cc:769] Sending connection context
0812 02:43:30.381746 (+ 2396us) client_negotiation.cc:240] Negotiation successful
0812 02:43:30.382058 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19061,"mutex_wait_us":142,"spinlock_wait_cycles":1408,"thread_start_us":13662,"threads_started":1}
I20260812 02:43:30.384310 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:30.332320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:60617 (local address 127.17.196.254:34567)
0812 02:43:30.351060 (+ 18740us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:30.351070 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:30.351092 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:43:30.351162 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:30.351170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:30.351246 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:30.351366 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:30.357583 (+ 6217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.359156 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.362885 (+ 3729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.363534 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.363695 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:30.372872 (+ 9177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:30.372879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:30.372884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:30.372960 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:30.378766 (+ 5806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:30.380300 (+ 1534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:30.380307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:30.380311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:30.380635 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:30.382069 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:30.382089 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:30.382093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:30.382552 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:30.382741 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:30.383370 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 02:43:30.383634 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18595,"thread_start_us":18438,"threads_started":1}
I20260812 02:43:30.384446 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:30.333832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.17.196.195:43795)
0812 02:43:30.348844 (+ 15012us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:30.348858 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:30.348901 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:43:30.349001 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:30.349025 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:30.349392 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:30.349643 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:30.353855 (+ 4212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.355459 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.358224 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.359593 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.359742 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:30.372073 (+ 12331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:30.372099 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:30.372124 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:30.372176 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:30.379019 (+ 6843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:30.380253 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:30.380265 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:30.380268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:30.380466 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:30.381056 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:30.381061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:30.381064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:30.382163 (+ 1099us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:30.382506 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:30.383535 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0812 02:43:30.383893 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14806,"thread_start_us":14636,"threads_started":1}
I20260812 02:43:30.386088 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:30.330370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.194:60617)
0812 02:43:30.350282 (+ 19912us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:30.350313 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:30.350480 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:30.351774 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:30.351792 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:30.355604 (+ 3812us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:30.357253 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:30.357272 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.360177 (+ 2905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.360183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:30.362680 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:30.362704 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.364588 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.364595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:30.364775 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:30.365566 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:30.365589 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:30.372587 (+ 6998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:30.379340 (+ 6753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:30.379347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:30.379351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:30.380162 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:30.381649 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:30.381654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:30.381657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:30.381924 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:30.382730 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:30.382735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:30.382892 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 02:43:30.383107 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 02:43:30.383574 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19659,"mutex_wait_us":928,"spinlock_wait_cycles":17664,"thread_start_us":7342,"threads_started":1}
I20260812 02:43:30.386349 26294 heartbeater.cc:347] Connected to a master server at 127.17.196.252:40163
I20260812 02:43:30.386456 25526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:30.386775 26294 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:30.387387 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:30.330370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.17.196.194:46171)
0812 02:43:30.339267 (+ 8897us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:30.339360 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:30.346593 (+ 7233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:30.352546 (+ 5953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:30.352555 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:30.353982 (+ 1427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:30.355049 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:30.355082 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.358628 (+ 3546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.358634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:30.361647 (+ 3013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:30.361663 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:30.363014 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:30.363024 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:30.363518 (+ 494us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:30.364838 (+ 1320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:30.364885 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:30.369792 (+ 4907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:30.376749 (+ 6957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:30.376758 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:30.376762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:30.377400 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:30.378130 (+ 730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:30.378135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:30.378138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:30.378403 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:30.379282 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:30.379287 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:30.379616 (+ 329us) client_negotiation.cc:769] Sending connection context
0812 02:43:30.380090 (+ 474us) client_negotiation.cc:240] Negotiation successful
0812 02:43:30.381721 (+ 1631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8415,"mutex_wait_us":173,"spinlock_wait_cycles":38656,"thread_start_us":22803,"threads_started":1}
I20260812 02:43:30.387533 26295 heartbeater.cc:347] Connected to a master server at 127.17.196.254:34567
I20260812 02:43:30.388219 26295 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:30.388329 26294 heartbeater.cc:511] Master 127.17.196.252:40163 requested a full tablet report, sending...
I20260812 02:43:30.390399 26296 heartbeater.cc:347] Connected to a master server at 127.17.196.253:46567
I20260812 02:43:30.390429 26295 heartbeater.cc:511] Master 127.17.196.254:34567 requested a full tablet report, sending...
I20260812 02:43:30.390573 26296 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:30.390965 25918 ts_manager.cc:194] Registered new tserver with Master: 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194:42369)
I20260812 02:43:30.391326 26296 heartbeater.cc:511] Master 127.17.196.253:46567 requested a full tablet report, sending...
I20260812 02:43:30.392340 25024 ts_manager.cc:194] Re-registered known tserver with Master: 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194:42369)
I20260812 02:43:30.393801 25024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:60617
I20260812 02:43:30.395705 25696 ts_manager.cc:194] Registered new tserver with Master: 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194:42369)
I20260812 02:43:30.482982 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 25388
I20260812 02:43:30.505138 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:43795
--local_ip_for_outbound_sockets=127.17.196.195
--tserver_master_addrs=127.17.196.254:34567
--webserver_port=42847
--webserver_interface=127.17.196.195
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab
--principal=oryx/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567,127.17.196.252:40163 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:43:30.724602 26129 heartbeater.cc:503] Master 127.17.196.254:34567 was elected leader, sending a full tablet report...
W20260812 02:43:30.886108 26310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:30.886508 26310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:30.886698 26310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:30.896297 26310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:30.896431 26310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:30.896703 26310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:30.908742 26310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab
--principal=oryx/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:43795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=42847
--tserver_master_addrs=127.17.196.254:34567,127.17.196.253:46567,127.17.196.252:40163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:30.910702 26310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:30.912400 26310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:30.928305 26317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:30.928339 26316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:30.928426 26319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:30.929898 26310 server_base.cc:1034] running on GCE node
I20260812 02:43:30.930687 26310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:30.931864 26310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:30.933125 26310 hybrid_clock.cc:648] HybridClock initialized: now 1786502610933052 us; error 58 us; skew 500 ppm
I20260812 02:43:30.933683 26310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:30 dist-test-slave-ss4n krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502610, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:30.942317 26310 init.cc:377] Logged in from keytab as oryx/127.17.196.195@KRBTEST.COM (short username oryx)
I20260812 02:43:30.944135 26310 webserver.cc:460] Webserver started at http://127.17.196.195:42847/ using document root <none> and password file <none>
I20260812 02:43:30.944888 26310 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:30.945060 26310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:30.951277 26310 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:43:30.954550 26326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:30.956215 26310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:43:30.956403 26310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "8b1cd287a35546828e6a22f690827612"
format_stamp: "Formatted at 2026-08-12 02:43:27 on dist-test-slave-ss4n"
server_key: "4f49ebede19857bd784533c0bd4e626b"
server_key_iv: "2f02d7143f3e9597498368d3a7a81bd6"
server_key_version: "encryptionkey@0"
I20260812 02:43:30.957634 26310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:30.976825 26310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:30.982533 26310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:30.982893 26310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:30.984287 26310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:30.985850 26310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:30.985944 26310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:30.986100 26310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:30.986207 26310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:31.029183 26310 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:43795
I20260812 02:43:31.029232 26457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:43795 every 8 connection(s)
I20260812 02:43:31.030871 26310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:31.035579 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 26310
Aug 12 02:43:31 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502610, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.253@KRBTEST.COM
Aug 12 02:43:31 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502610, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:31.070042 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.034689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:48551 (local address 127.17.196.253:46567)
0812 02:43:31.035445 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.035452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.036891 (+ 1439us) server_negotiation.cc:407] Connection header received
0812 02:43:31.040748 (+ 3857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.040758 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.040895 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.041006 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.044689 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.046401 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.050302 (+ 3901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.050752 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.050913 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.059415 (+ 8502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.059424 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.059432 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.059489 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.064528 (+ 5039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.066045 (+ 1517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.066057 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.066060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.066216 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.066800 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.066806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.066810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.067241 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.067409 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.069122 (+ 1713us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.069394 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":760,"threads_started":1}
I20260812 02:43:31.078384 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.035737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.17.196.195:48551)
0812 02:43:31.036712 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:31.036764 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:31.040217 (+ 3453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:31.041918 (+ 1701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:31.041936 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:31.043116 (+ 1180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:31.044339 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.044378 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.046749 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.046758 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.050125 (+ 3367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.050138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.051052 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.051062 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.051486 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.052553 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:31.052585 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:31.058551 (+ 5966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:31.064847 (+ 6296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:31.064859 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:31.064883 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:31.065856 (+ 973us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:31.066383 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:31.066389 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:31.066394 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:31.066638 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:31.067588 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:31.067599 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:31.068309 (+ 710us) client_negotiation.cc:769] Sending connection context
0812 02:43:31.070313 (+ 2004us) client_negotiation.cc:240] Negotiation successful
0812 02:43:31.071033 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":509,"threads_started":1}
I20260812 02:43:31.081662 26460 heartbeater.cc:347] Connected to a master server at 127.17.196.253:46567
I20260812 02:43:31.082113 26460 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:31.083603 26464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.034322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:33127 (local address 127.17.196.254:34567)
0812 02:43:31.035210 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.035218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.049473 (+ 14255us) server_negotiation.cc:407] Connection header received
0812 02:43:31.050018 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.050024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.050117 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.050232 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.053253 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.054838 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.059090 (+ 4252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.060124 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.060331 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.065985 (+ 5654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.065996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.066005 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.066071 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.072379 (+ 6308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.078337 (+ 5958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.078348 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.078354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.078613 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.079336 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.079344 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.079349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.080018 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.080244 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.082726 (+ 2482us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.083075 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":635,"threads_started":1}
I20260812 02:43:31.083622 26460 heartbeater.cc:511] Master 127.17.196.253:46567 requested a full tablet report, sending...
I20260812 02:43:31.086009 25696 ts_manager.cc:194] Registered new tserver with Master: 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795)
I20260812 02:43:31.087226 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.035737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.195:33127)
0812 02:43:31.049328 (+ 13591us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:31.049359 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:31.049853 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:31.051156 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:31.051161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:31.052012 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:31.053043 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.053059 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.055451 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.055458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.058867 (+ 3409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.058881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.060605 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.060671 (+ 66us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.060993 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.061866 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:31.061903 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:31.065682 (+ 3779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:31.072707 (+ 7025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:31.072714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:31.072719 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:31.078114 (+ 5395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:31.078942 (+ 828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:31.078950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:31.078957 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:31.079201 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:31.081937 (+ 2736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:31.081942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:31.082346 (+ 404us) client_negotiation.cc:769] Sending connection context
0812 02:43:31.082532 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 02:43:31.082976 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13391,"mutex_wait_us":30,"thread_start_us":24613,"threads_started":1}
I20260812 02:43:31.088341 26459 heartbeater.cc:347] Connected to a master server at 127.17.196.254:34567
I20260812 02:43:31.088614 26459 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:31.090013 26459 heartbeater.cc:511] Master 127.17.196.254:34567 requested a full tablet report, sending...
I20260812 02:43:31.092490 25024 ts_manager.cc:194] Re-registered known tserver with Master: 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795)
I20260812 02:43:31.094369 25024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:33127
Aug 12 02:43:31 dist-test-slave-ss4n krb5kdc[24975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502610, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.252@KRBTEST.COM
I20260812 02:43:31.111585 26462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.034268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:39345 (local address 127.17.196.252:40163)
0812 02:43:31.034858 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.034868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.059874 (+ 25006us) server_negotiation.cc:407] Connection header received
0812 02:43:31.060237 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.060245 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.061627 (+ 1382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.061768 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.064241 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.067002 (+ 2761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.086541 (+ 19539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.087830 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.087989 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.098282 (+ 10293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.098296 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.098306 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.098375 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.105824 (+ 7449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.107303 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.107309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.107313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.107537 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.108026 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.108034 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.108039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.108512 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.108762 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.110553 (+ 1791us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.110814 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":302,"threads_started":1}
I20260812 02:43:31.112061 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.036107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.17.196.195:39345)
0812 02:43:31.059746 (+ 23639us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:31.059771 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:31.060001 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:31.062200 (+ 2199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:31.062215 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:31.063445 (+ 1230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:31.064012 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.064029 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.067204 (+ 3175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.067210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.086310 (+ 19100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.086332 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.090335 (+ 4003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.090353 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.090768 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.092543 (+ 1775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:31.092697 (+ 154us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:31.097826 (+ 5129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:31.106110 (+ 8284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:31.106118 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:31.106124 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:31.107100 (+ 976us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:31.107701 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:31.107708 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:31.107712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:31.107878 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:31.108877 (+ 999us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:31.108882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:31.109225 (+ 343us) client_negotiation.cc:769] Sending connection context
0812 02:43:31.109433 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 02:43:31.109936 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23454,"thread_start_us":13382,"threads_started":1}
I20260812 02:43:31.113372 26458 heartbeater.cc:347] Connected to a master server at 127.17.196.252:40163
I20260812 02:43:31.113658 26458 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:31.114200 26458 heartbeater.cc:511] Master 127.17.196.252:40163 requested a full tablet report, sending...
I20260812 02:43:31.115477 25918 ts_manager.cc:194] Registered new tserver with Master: 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795)
I20260812 02:43:31.126756 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.075775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.17.196.253:46567)
0812 02:43:31.076025 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.076034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.102420 (+ 26386us) server_negotiation.cc:407] Connection header received
0812 02:43:31.102657 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.102663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.102780 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.102876 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.105294 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.106623 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.110014 (+ 3391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.110916 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.111036 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.116614 (+ 5578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.116623 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.116628 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.116671 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.122280 (+ 5609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.123330 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.123335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.123339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.123538 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.124071 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.124080 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.124086 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.124607 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.124866 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.125754 (+ 888us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.126125 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:43:31.132256 26464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.075802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.17.196.254:34567)
0812 02:43:31.083934 (+ 8132us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.083942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.096341 (+ 12399us) server_negotiation.cc:407] Connection header received
0812 02:43:31.107259 (+ 10918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.107268 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.107400 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.107513 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.114590 (+ 7077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.115705 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.118281 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.118861 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.119026 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.123007 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.123013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.123018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.123063 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.128295 (+ 5232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.129360 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.129368 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.129374 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.129646 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.130292 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.130296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.130298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.130724 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.131128 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.131520 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.131864 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7978,"thread_start_us":26808,"threads_started":1}
I20260812 02:43:31.132365 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.076741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47520 (local address 127.17.196.252:40163)
0812 02:43:31.096381 (+ 19640us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.096399 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.106509 (+ 10110us) server_negotiation.cc:407] Connection header received
0812 02:43:31.106574 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.106579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.106661 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.106763 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.111301 (+ 4538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.113366 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.116132 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.117334 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.117468 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.122585 (+ 5117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.122593 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.122598 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.122655 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.128094 (+ 5439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.129169 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.129175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.129178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.129649 (+ 471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.130140 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.130146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.130150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.130571 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.130832 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.131382 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.131681 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16156,"thread_start_us":15976,"threads_started":1}
I20260812 02:43:31.138094 25024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:43:31.148386 25024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:31.197633 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.176122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:42369 (local address 127.0.0.1:47486)
0812 02:43:31.182594 (+ 6472us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:31.182612 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:31.182741 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:31.184909 (+ 2168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:31.184915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:31.184950 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:31.185730 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.185760 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.189002 (+ 3242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.189008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.192627 (+ 3619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.192655 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.196237 (+ 3582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.196246 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.196572 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.196630 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 02:43:31.196779 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 02:43:31.197027 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6280,"thread_start_us":3338,"threads_started":1}
I20260812 02:43:31.198532 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.176521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.17.196.194:42369)
0812 02:43:31.184177 (+ 7656us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.184185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.184210 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:43:31.184290 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.184295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.184549 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.185176 (+ 627us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:31.186099 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.188785 (+ 2686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.192919 (+ 4134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.195291 (+ 2372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.195652 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.195877 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.197675 (+ 1798us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.197998 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7489,"thread_start_us":7198,"threads_started":1}
I20260812 02:43:31.201097 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.177799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:43795 (local address 127.0.0.1:35872)
0812 02:43:31.183403 (+ 5604us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:31.183422 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:31.183547 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:31.184543 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:31.184548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:31.184576 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:31.188319 (+ 3743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.188342 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.191839 (+ 3497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.191846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.196238 (+ 4392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.196254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.199781 (+ 3527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.199792 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.200086 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.200125 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:43:31.200257 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 02:43:31.200445 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5448,"mutex_wait_us":34,"spinlock_wait_cycles":1408,"thread_start_us":5296,"threads_started":1}
I20260812 02:43:31.201220 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.175714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59188 (local address 127.17.196.193:34391)
0812 02:43:31.186398 (+ 10684us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.186409 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.186453 (+ 44us) server_negotiation.cc:407] Connection header received
0812 02:43:31.186617 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.186637 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.186984 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.187384 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:31.189250 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.192212 (+ 2962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.196042 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.199669 (+ 3627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.199950 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.200148 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.200592 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.200782 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10452,"thread_start_us":10241,"threads_started":1}
I20260812 02:43:31.202209 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.178665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.17.196.195:43795)
0812 02:43:31.180273 (+ 1608us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.180287 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.183568 (+ 3281us) server_negotiation.cc:407] Connection header received
0812 02:43:31.183733 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.183746 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.184154 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.184552 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:31.189083 (+ 4531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.191658 (+ 2575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.196532 (+ 4874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.199244 (+ 2712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.199391 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.199539 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.201534 (+ 1995us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.201755 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1355,"thread_start_us":1026,"threads_started":1}
I20260812 02:43:31.201514 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.175404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:34391 (local address 127.0.0.1:59188)
0812 02:43:31.177935 (+ 2531us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:31.177963 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:31.181635 (+ 3672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:31.187490 (+ 5855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:31.187494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:31.187518 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:31.188893 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.188921 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.192436 (+ 3515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.192442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.195774 (+ 3332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.195795 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.199998 (+ 4203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.200006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.200365 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.200399 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:43:31.200522 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 02:43:31.200675 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2001,"mutex_wait_us":69,"thread_start_us":6756,"threads_started":1}
I20260812 02:43:31.204154 26221 tablet_service.cc:1467] Processing CreateTablet for tablet 53019ddf481e48489a2fa90a36d20274 (DEFAULT_TABLE table=test-table [id=5f9553c5d24c47b6a25db7950b1ff735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:31.207520 26221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53019ddf481e48489a2fa90a36d20274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:31.208850 26383 tablet_service.cc:1467] Processing CreateTablet for tablet 53019ddf481e48489a2fa90a36d20274 (DEFAULT_TABLE table=test-table [id=5f9553c5d24c47b6a25db7950b1ff735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:31.209427 26052 tablet_service.cc:1467] Processing CreateTablet for tablet 53019ddf481e48489a2fa90a36d20274 (DEFAULT_TABLE table=test-table [id=5f9553c5d24c47b6a25db7950b1ff735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:31.210872 26383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53019ddf481e48489a2fa90a36d20274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:31.211127 26052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53019ddf481e48489a2fa90a36d20274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:31.237224 26497 tablet_bootstrap.cc:492] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Bootstrap starting.
I20260812 02:43:31.243203 26498 tablet_bootstrap.cc:492] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Bootstrap starting.
I20260812 02:43:31.245625 26496 tablet_bootstrap.cc:492] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Bootstrap starting.
I20260812 02:43:31.246704 26498 tablet_bootstrap.cc:654] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:31.246701 26497 tablet_bootstrap.cc:654] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:31.248622 26498 log.cc:824] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:31.249207 26497 log.cc:824] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:31.251595 26496 tablet_bootstrap.cc:654] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:31.253731 26496 log.cc:824] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:31.255290 26497 tablet_bootstrap.cc:492] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: No bootstrap required, opened a new log
I20260812 02:43:31.255373 26498 tablet_bootstrap.cc:492] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: No bootstrap required, opened a new log
I20260812 02:43:31.255784 26497 ts_tablet_manager.cc:1397] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.002s
I20260812 02:43:31.255889 26498 ts_tablet_manager.cc:1397] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.010s
I20260812 02:43:31.257545 26496 tablet_bootstrap.cc:492] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: No bootstrap required, opened a new log
I20260812 02:43:31.257833 26496 ts_tablet_manager.cc:1397] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260812 02:43:31.270846 26496 raft_consensus.cc:348] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.271390 26496 raft_consensus.cc:374] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:31.271826 26496 raft_consensus.cc:729] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af9696aa3b441cabaf65f148ff973f2, State: Initialized, Role: FOLLOWER
I20260812 02:43:31.272082 26498 raft_consensus.cc:348] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.272485 26497 raft_consensus.cc:348] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.272867 26498 raft_consensus.cc:374] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:31.272989 26498 raft_consensus.cc:729] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1cd287a35546828e6a22f690827612, State: Initialized, Role: FOLLOWER
I20260812 02:43:31.273110 26497 raft_consensus.cc:374] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:31.273049 26496 consensus_queue.cc:260] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.273315 26497 raft_consensus.cc:729] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8a58be1415e4d268335e44dec2c1baf, State: Initialized, Role: FOLLOWER
I20260812 02:43:31.273870 26498 consensus_queue.cc:260] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.274250 26497 consensus_queue.cc:260] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.275980 26496 ts_tablet_manager.cc:1428] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Time spent starting tablet: real 0.018s user 0.011s sys 0.005s
I20260812 02:43:31.276552 26295 heartbeater.cc:503] Master 127.17.196.254:34567 was elected leader, sending a full tablet report...
I20260812 02:43:31.280715 26497 ts_tablet_manager.cc:1428] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Time spent starting tablet: real 0.025s user 0.005s sys 0.017s
I20260812 02:43:31.287284 26459 heartbeater.cc:503] Master 127.17.196.254:34567 was elected leader, sending a full tablet report...
I20260812 02:43:31.290855 26498 ts_tablet_manager.cc:1428] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Time spent starting tablet: real 0.035s user 0.024s sys 0.006s
W20260812 02:43:31.353147 26298 tablet.cc:2369] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:31.386677 26506 raft_consensus.cc:482] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:31.386994 26506 raft_consensus.cc:504] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.390439 26506 leader_election.cc:288] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194:42369), 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795)
I20260812 02:43:31.399848 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.389948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:43795 (local address 127.17.196.193:45749)
0812 02:43:31.391061 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:31.391086 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:31.391306 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:31.391995 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:31.392001 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:31.392036 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:31.392675 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.392690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.394453 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.394459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.397083 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.397100 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.398974 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.398981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.399166 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.399202 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 02:43:31.399313 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 02:43:31.399435 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"thread_start_us":558,"threads_started":1}
I20260812 02:43:31.400445 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.390053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:45749 (local address 127.17.196.195:43795)
0812 02:43:31.390371 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.390380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.391237 (+ 857us) server_negotiation.cc:407] Connection header received
0812 02:43:31.391691 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.391698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.391811 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.391931 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:31.392869 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.394229 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.397281 (+ 3052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.398735 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.398887 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.398950 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.399423 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.399898 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:43:31.400698 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.390034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:39079 (local address 127.17.196.194:42369)
0812 02:43:31.390332 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.390339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.391692 (+ 1353us) server_negotiation.cc:407] Connection header received
0812 02:43:31.391995 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.392024 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.392127 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.392260 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:31.393393 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.394804 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.397892 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.399001 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.399190 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.399266 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.400362 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.400472 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260812 02:43:31.400882 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.390365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:42369 (local address 127.17.196.193:39079)
0812 02:43:31.391515 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:31.391533 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:31.391753 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:31.392315 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:31.392320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:31.392349 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:31.393037 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.393067 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.395054 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.395059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.397663 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:31.397680 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.399831 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.399847 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:31.400174 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.400214 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:43:31.400312 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 02:43:31.400445 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"thread_start_us":685,"threads_started":1}
I20260812 02:43:31.401764 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53019ddf481e48489a2fa90a36d20274" candidate_uuid: "f8a58be1415e4d268335e44dec2c1baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b1cd287a35546828e6a22f690827612" is_pre_election: true
I20260812 02:43:31.402168 26243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53019ddf481e48489a2fa90a36d20274" candidate_uuid: "f8a58be1415e4d268335e44dec2c1baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af9696aa3b441cabaf65f148ff973f2" is_pre_election: true
I20260812 02:43:31.402549 26404 raft_consensus.cc:2454] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8a58be1415e4d268335e44dec2c1baf in term 0.
I20260812 02:43:31.403013 26243 raft_consensus.cc:2454] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8a58be1415e4d268335e44dec2c1baf in term 0.
I20260812 02:43:31.403728 25995 leader_election.cc:302] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b1cd287a35546828e6a22f690827612, f8a58be1415e4d268335e44dec2c1baf; no voters:
I20260812 02:43:31.404364 26506 raft_consensus.cc:2781] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:31.404569 26506 raft_consensus.cc:482] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:31.404809 26506 raft_consensus.cc:3037] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:31.408295 26506 raft_consensus.cc:504] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.409348 26506 leader_election.cc:288] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [CANDIDATE]: Term 1 election: Requested vote from peers 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194:42369), 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795)
I20260812 02:43:31.410166 26243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53019ddf481e48489a2fa90a36d20274" candidate_uuid: "f8a58be1415e4d268335e44dec2c1baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4af9696aa3b441cabaf65f148ff973f2"
I20260812 02:43:31.410198 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53019ddf481e48489a2fa90a36d20274" candidate_uuid: "f8a58be1415e4d268335e44dec2c1baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b1cd287a35546828e6a22f690827612"
I20260812 02:43:31.410450 26243 raft_consensus.cc:3037] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:31.410560 26404 raft_consensus.cc:3037] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:31.415684 26243 raft_consensus.cc:2454] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8a58be1415e4d268335e44dec2c1baf in term 1.
I20260812 02:43:31.415757 26404 raft_consensus.cc:2454] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8a58be1415e4d268335e44dec2c1baf in term 1.
I20260812 02:43:31.416553 25996 leader_election.cc:302] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af9696aa3b441cabaf65f148ff973f2, f8a58be1415e4d268335e44dec2c1baf; no voters:
I20260812 02:43:31.417052 26506 raft_consensus.cc:2781] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:31.417833 26506 raft_consensus.cc:686] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 1 LEADER]: Becoming Leader. State: Replica: f8a58be1415e4d268335e44dec2c1baf, State: Running, Role: LEADER
I20260812 02:43:31.418485 26506 consensus_queue.cc:237] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:31.424988 25024 catalog_manager.cc:5495] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf reported cstate change: term changed from 0 to 1, leader changed from <none> to f8a58be1415e4d268335e44dec2c1baf (127.17.196.193). New cstate: current_term: 1 leader_uuid: "f8a58be1415e4d268335e44dec2c1baf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } health_report { overall_health: UNKNOWN } } }
W20260812 02:43:31.455042 26136 tablet.cc:2369] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:31.503784 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.497116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.17.196.193:34391)
0812 02:43:31.497367 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.497372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.497390 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:43:31.497585 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.497589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.497677 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.497806 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:31.498247 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.499317 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.500799 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.501591 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.501716 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.502329 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:31.503173 (+ 844us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:31.503394 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.503445 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.503529 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 02:43:31.515439 26404 raft_consensus.cc:1261] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 1 FOLLOWER]: Refusing update from remote peer f8a58be1415e4d268335e44dec2c1baf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:31.515574 26243 raft_consensus.cc:1261] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 FOLLOWER]: Refusing update from remote peer f8a58be1415e4d268335e44dec2c1baf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:31.516767 26510 consensus_queue.cc:1035] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:31.517634 26506 consensus_queue.cc:1035] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 02:43:31.534508 26461 tablet.cc:2369] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:31.545032 26514 mvcc.cc:205] Tried to move back new op lower bound from 7317514696753623040 to 7317514696374243328. Current Snapshot: MvccSnapshot[applied={T|T < 7317514696753623040}]
I20260812 02:43:31.562911 25024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:31.563318 25024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:31.572631 25024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2dce354ed6604f9dba212f727ac21f28: Sending DeleteTablet for 3 replicas of tablet 53019ddf481e48489a2fa90a36d20274
I20260812 02:43:31.574208 26221 tablet_service.cc:1514] Processing DeleteTablet for tablet 53019ddf481e48489a2fa90a36d20274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:31 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:47486
I20260812 02:43:31.574097 26052 tablet_service.cc:1514] Processing DeleteTablet for tablet 53019ddf481e48489a2fa90a36d20274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:31 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:59188
I20260812 02:43:31.574517 26383 tablet_service.cc:1514] Processing DeleteTablet for tablet 53019ddf481e48489a2fa90a36d20274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:31 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:35872
I20260812 02:43:31.575755 26526 tablet_replica.cc:330] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: stopping tablet replica
I20260812 02:43:31.576311 26526 raft_consensus.cc:2229] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:31.576908 26526 raft_consensus.cc:2258] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:31.581770 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 24988
I20260812 02:43:31.581751 26526 ts_tablet_manager.cc:1905] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:31.583588 25057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:31.593237 26527 tablet_replica.cc:330] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: stopping tablet replica
I20260812 02:43:31.594019 26527 raft_consensus.cc:2229] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:43:31.595392 26527 raft_consensus.cc:2258] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:31.604290 26527 ts_tablet_manager.cc:1905] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:31.604876 26528 tablet_replica.cc:330] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: stopping tablet replica
I20260812 02:43:31.605504 26528 raft_consensus.cc:2229] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:31.606348 26528 raft_consensus.cc:2258] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:31.601447 26526 ts_tablet_manager.cc:1918] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:31.609939 26526 log.cc:1196] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/53019ddf481e48489a2fa90a36d20274
I20260812 02:43:31.610774 26528 ts_tablet_manager.cc:1905] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:31.611115 26526 ts_tablet_manager.cc:1939] T 53019ddf481e48489a2fa90a36d20274 P 4af9696aa3b441cabaf65f148ff973f2: Deleting consensus metadata
I20260812 02:43:31.618878 26527 ts_tablet_manager.cc:1918] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:31.619136 26527 log.cc:1196] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/53019ddf481e48489a2fa90a36d20274
I20260812 02:43:31.620049 26527 ts_tablet_manager.cc:1939] T 53019ddf481e48489a2fa90a36d20274 P f8a58be1415e4d268335e44dec2c1baf: Deleting consensus metadata
I20260812 02:43:31.620407 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.578460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.17.196.253:46567)
0812 02:43:31.578768 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.578778 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.579230 (+ 452us) server_negotiation.cc:407] Connection header received
0812 02:43:31.579444 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.579450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.579526 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.579642 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.583731 (+ 4089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.585248 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.588448 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.589389 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.589538 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.595589 (+ 6051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.595599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.595607 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.595674 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.601457 (+ 5783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.617367 (+ 15910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.617373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.617379 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.617642 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.618155 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.618160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.618163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.618591 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.618949 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.619685 (+ 736us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.619969 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260812 02:43:31.623430 26528 ts_tablet_manager.cc:1918] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:31.623839 26528 log.cc:1196] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/53019ddf481e48489a2fa90a36d20274
I20260812 02:43:31.624842 26528 ts_tablet_manager.cc:1939] T 53019ddf481e48489a2fa90a36d20274 P 8b1cd287a35546828e6a22f690827612: Deleting consensus metadata
I20260812 02:43:31.648766 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.579873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.17.196.252:40163)
0812 02:43:31.580073 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.580080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.621698 (+ 41618us) server_negotiation.cc:407] Connection header received
0812 02:43:31.621993 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.621998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.622090 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.622201 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.629742 (+ 7541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.631001 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.633530 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.634432 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.634553 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.638898 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.638907 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.638924 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.638974 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.643854 (+ 4880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.645240 (+ 1386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.645247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.645251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.645601 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.646227 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.646232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.646236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.646736 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.647114 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.647803 (+ 689us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.648160 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 02:43:31.751173 25005 catalog_manager.cc:4841] TS 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194:42369): tablet 53019ddf481e48489a2fa90a36d20274 (table test-table [id=5f9553c5d24c47b6a25db7950b1ff735]) successfully deleted
I20260812 02:43:31.752381 25006 catalog_manager.cc:4841] TS f8a58be1415e4d268335e44dec2c1baf (127.17.196.193:34391): tablet 53019ddf481e48489a2fa90a36d20274 (table test-table [id=5f9553c5d24c47b6a25db7950b1ff735]) successfully deleted
I20260812 02:43:31.752381 25004 catalog_manager.cc:4841] TS 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795): tablet 53019ddf481e48489a2fa90a36d20274 (table test-table [id=5f9553c5d24c47b6a25db7950b1ff735]) successfully deleted
I20260812 02:43:31.753093 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 24988
I20260812 02:43:31.814958 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.788687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.17.196.252:40163)
0812 02:43:31.788902 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.788908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.794445 (+ 5537us) server_negotiation.cc:407] Connection header received
0812 02:43:31.794829 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.794836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.794951 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.795046 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.798585 (+ 3539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.799856 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.802942 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.803722 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.803868 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.807087 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.807095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.807100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.807151 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.811309 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.812414 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.812421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.812425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.812602 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.813115 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.813123 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.813126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.813603 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.813889 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.814317 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.814603 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 02:43:31.818553 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:31.787916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.17.196.253:46567)
0812 02:43:31.788508 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:31.788516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:31.798373 (+ 9857us) server_negotiation.cc:407] Connection header received
0812 02:43:31.798529 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:31.798534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:31.798634 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:31.798777 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:31.802847 (+ 4070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.803757 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.806116 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:31.806802 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:31.806956 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:31.810759 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:31.810764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:31.810768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:31.810807 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:31.814331 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.815924 (+ 1593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.815931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.815935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.816164 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:31.816656 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:31.816662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:31.816666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:31.817153 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:31.817431 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:31.817875 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 02:43:31.818165 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260812 02:43:32.549039 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:32.548022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.193:49851)
0812 02:43:32.548621 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:32.548799 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":324,"threads_started":1}
W20260812 02:43:32.549585 26129 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:34567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
I20260812 02:43:32.554857 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:32.553884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.195:44833)
0812 02:43:32.554374 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:32.554566 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":252,"threads_started":1}
W20260812 02:43:32.555325 26459 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:34567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
I20260812 02:43:32.556636 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:32.555776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.194:37427)
0812 02:43:32.556306 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:32.556421 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":278,"threads_started":1}
W20260812 02:43:32.557080 26295 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:34567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
I20260812 02:43:33.071225 26541 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2dce354ed6604f9dba212f727ac21f28)
I20260812 02:43:33.071547 26541 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } }
I20260812 02:43:33.072292 26542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2dce354ed6604f9dba212f727ac21f28)
I20260812 02:43:33.072567 26542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } }
I20260812 02:43:33.073194 26541 leader_election.cc:288] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2dce354ed6604f9dba212f727ac21f28 (127.17.196.254:34567), 60c87d55fe1943f99e0a51376de660c9 (127.17.196.252:40163)
I20260812 02:43:33.074405 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.073321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41666)
0812 02:43:33.073910 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.074159 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":392,"threads_started":1}
W20260812 02:43:33.074762 25682 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
I20260812 02:43:33.080051 26542 leader_election.cc:288] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2dce354ed6604f9dba212f727ac21f28 (127.17.196.254:34567), b1f1396065e740869a40a86ff99bf02f (127.17.196.253:46567)
I20260812 02:43:33.082966 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.081286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41682)
0812 02:43:33.082269 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.082537 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":819}
I20260812 02:43:33.083189 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.073953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41670)
0812 02:43:33.082601 (+ 8648us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.082882 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7754,"thread_start_us":7653,"threads_started":1}
W20260812 02:43:33.083614 25902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
W20260812 02:43:33.087512 25682 leader_election.cc:334] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2dce354ed6604f9dba212f727ac21f28 (127.17.196.254:34567): Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
I20260812 02:43:33.089609 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.088900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41698)
0812 02:43:33.089127 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.089301 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260812 02:43:33.089794 25902 leader_election.cc:334] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2dce354ed6604f9dba212f727ac21f28 (127.17.196.254:34567): Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
I20260812 02:43:33.090057 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.073328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40163 (local address 127.0.0.1:56648)
0812 02:43:33.080190 (+ 6862us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.080220 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:33.080408 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:33.081691 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:33.081700 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:33.081731 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:33.081966 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.081980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.084068 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.084077 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.087090 (+ 3013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.087116 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.089140 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.089146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.089297 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.089326 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:43:33.089393 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 02:43:33.089609 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6745,"mutex_wait_us":216,"thread_start_us":8440,"threads_started":1}
I20260812 02:43:33.090574 26546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.073972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56648 (local address 127.17.196.252:40163)
0812 02:43:33.081207 (+ 7235us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.081214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.081237 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:43:33.081323 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.081328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.081412 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.082161 (+ 749us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:33.082419 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.083873 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.087380 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.088934 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.089940 (+ 1006us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.090055 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.090116 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.090259 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7104,"thread_start_us":6979,"threads_started":1}
I20260812 02:43:33.091288 25930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1f1396065e740869a40a86ff99bf02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "60c87d55fe1943f99e0a51376de660c9" is_pre_election: true
I20260812 02:43:33.091701 25930 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f1396065e740869a40a86ff99bf02f in term 1.
I20260812 02:43:33.092245 25683 leader_election.cc:302] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60c87d55fe1943f99e0a51376de660c9, b1f1396065e740869a40a86ff99bf02f; no voters: 2dce354ed6604f9dba212f727ac21f28
I20260812 02:43:33.092657 26541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 02:43:33.092811 26541 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2dce354ed6604f9dba212f727ac21f28)
I20260812 02:43:33.092948 26541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:43:33.095413 26549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.081992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.17.196.253:46567)
0812 02:43:33.087713 (+ 5721us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.087722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.087756 (+ 34us) server_negotiation.cc:407] Connection header received
0812 02:43:33.087836 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.087844 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.087941 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.088485 (+ 544us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:33.089185 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.090535 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.092848 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.094303 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.094492 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.094594 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.094853 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.095013 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5537,"thread_start_us":5263,"threads_started":1}
I20260812 02:43:33.095319 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.081930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:46567 (local address 127.0.0.1:43034)
0812 02:43:33.083420 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.083438 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:33.083551 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:33.088537 (+ 4986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:33.088545 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:33.088617 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:33.088917 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.088927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.090694 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.090700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.092636 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.092654 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.094511 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.094516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.094682 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.094709 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 02:43:33.094797 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 02:43:33.094977 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1399,"thread_start_us":6128,"threads_started":1}
I20260812 02:43:33.096266 25708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60c87d55fe1943f99e0a51376de660c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b1f1396065e740869a40a86ff99bf02f" is_pre_election: true
I20260812 02:43:33.096400 26541 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } }
I20260812 02:43:33.096875 25708 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60c87d55fe1943f99e0a51376de660c9 in current term 2: Already voted for candidate b1f1396065e740869a40a86ff99bf02f in this term.
I20260812 02:43:33.097319 26541 leader_election.cc:288] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [CANDIDATE]: Term 2 election: Requested vote from peers 2dce354ed6604f9dba212f727ac21f28 (127.17.196.254:34567), 60c87d55fe1943f99e0a51376de660c9 (127.17.196.252:40163)
I20260812 02:43:33.097868 25930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1f1396065e740869a40a86ff99bf02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "60c87d55fe1943f99e0a51376de660c9"
I20260812 02:43:33.097945 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.097448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41700)
0812 02:43:33.097650 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.097780 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 02:43:33.097973 25899 leader_election.cc:302] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60c87d55fe1943f99e0a51376de660c9; no voters: 2dce354ed6604f9dba212f727ac21f28, b1f1396065e740869a40a86ff99bf02f
I20260812 02:43:33.098136 25930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:43:33.099152 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.098750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41714)
0812 02:43:33.098914 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.099006 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 02:43:33.099334 25682 leader_election.cc:334] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2dce354ed6604f9dba212f727ac21f28 (127.17.196.254:34567): Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
I20260812 02:43:33.100489 25930 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f1396065e740869a40a86ff99bf02f in term 2.
I20260812 02:43:33.100680 26542 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 02:43:33.100984 25683 leader_election.cc:302] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60c87d55fe1943f99e0a51376de660c9, b1f1396065e740869a40a86ff99bf02f; no voters: 2dce354ed6604f9dba212f727ac21f28
I20260812 02:43:33.101265 26541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:43:33.101795 26541 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [term 2 LEADER]: Becoming Leader. State: Replica: b1f1396065e740869a40a86ff99bf02f, State: Running, Role: LEADER
I20260812 02:43:33.102223 26541 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } }
I20260812 02:43:33.104460 26552 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1f1396065e740869a40a86ff99bf02f. Latest consensus state: current_term: 2 leader_uuid: "b1f1396065e740869a40a86ff99bf02f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:33.104964 26552 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:33.105860 26554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:33.109822 26554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:33.110656 26554 catalog_manager.cc:1259] Loaded cluster ID: 84bd3a9d86e94265ae1b366d3829c551
I20260812 02:43:33.110728 26554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:33.111948 26554 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:33.112789 26554 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Loaded TSK: 0
I20260812 02:43:33.113713 26554 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:43:33.125584 25695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:43:33.134188 25695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:33.139302 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.138599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41730)
0812 02:43:33.138948 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.139088 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260812 02:43:33.139361 25930 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [term 2 FOLLOWER]: Refusing update from remote peer b1f1396065e740869a40a86ff99bf02f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 02:43:33.140360 26552 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 02:43:33.140724 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.140134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41744)
0812 02:43:33.140369 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.140471 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260812 02:43:33.140957 25682 consensus_peers.cc:487] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f -> Peer 2dce354ed6604f9dba212f727ac21f28 (127.17.196.254:34567): Couldn't send request to peer 2dce354ed6604f9dba212f727ac21f28. Status: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 02:43:33.143702 26542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1f1396065e740869a40a86ff99bf02f. Latest consensus state: current_term: 2 leader_uuid: "b1f1396065e740869a40a86ff99bf02f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:33.144029 26542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:43:33.145942 26552 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b1f1396065e740869a40a86ff99bf02f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:33.146293 26552 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:33.156258 26542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b1f1396065e740869a40a86ff99bf02f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2dce354ed6604f9dba212f727ac21f28" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34567 } } peers { permanent_uuid: "b1f1396065e740869a40a86ff99bf02f" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 46567 } attrs { promote: false } } peers { permanent_uuid: "60c87d55fe1943f99e0a51376de660c9" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40163 } attrs { promote: false } } }
I20260812 02:43:33.156620 26542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c87d55fe1943f99e0a51376de660c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:43:33.171332 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.161614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:34391 (local address 127.0.0.1:55342)
0812 02:43:33.161874 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.161896 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:33.162046 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:33.164663 (+ 2617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:33.164667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:33.164689 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:33.164837 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.164845 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.166588 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.166594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.168717 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.168740 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.170482 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.170489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.170699 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.170739 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:43:33.170841 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 02:43:33.170993 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260812 02:43:33.171427 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.161939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.17.196.193:34391)
0812 02:43:33.164237 (+ 2298us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.164246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.164269 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:43:33.164358 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.164366 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.164475 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.164597 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:33.164992 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.166457 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.168974 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.170270 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.170491 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.170581 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.170897 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.171045 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2122,"thread_start_us":1804,"threads_started":1}
I20260812 02:43:33.172212 26052 tablet_service.cc:1467] Processing CreateTablet for tablet cb72ce347abc4e5a84bdff409bd05472 (DEFAULT_TABLE table=test-table [id=91d1dc9321684c87a9ba0cf0debd9154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:33.172792 26052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb72ce347abc4e5a84bdff409bd05472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:33.173098 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.162251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:42369 (local address 127.0.0.1:35834)
0812 02:43:33.162453 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.162472 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:33.162591 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:33.166938 (+ 4347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:33.166943 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:33.166968 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:33.167404 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.167421 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.168793 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.168801 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.170309 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.170321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.172283 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.172288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.172486 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.172528 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 02:43:33.172600 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 02:43:33.172768 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":25}
I20260812 02:43:33.173628 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.162696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:43795 (local address 127.0.0.1:45266)
0812 02:43:33.163450 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.163474 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:33.163627 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:33.166138 (+ 2511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:33.166142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:33.166167 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:33.166529 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.166541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.167949 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.167954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.170549 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.170565 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.172248 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.172257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.172624 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.172653 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:43:33.172731 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 02:43:33.172840 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":209,"thread_start_us":727,"threads_started":1}
I20260812 02:43:33.174067 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.163212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.17.196.195:43795)
0812 02:43:33.165852 (+ 2640us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.165857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.165873 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:43:33.165931 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.165934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.165988 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.166111 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:33.166689 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.167784 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.170704 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.171979 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.172136 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.172223 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.172942 (+ 719us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.173099 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2539,"thread_start_us":2453,"threads_started":1}
I20260812 02:43:33.174325 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.162913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.17.196.194:42369)
0812 02:43:33.166629 (+ 3716us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.166638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.166659 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:43:33.166735 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.166738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.166795 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.166908 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:33.167609 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.168648 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.170554 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.172017 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.173721 (+ 1704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.173815 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.173871 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.174012 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3575,"thread_start_us":3455,"threads_started":1}
I20260812 02:43:33.175102 26383 tablet_service.cc:1467] Processing CreateTablet for tablet cb72ce347abc4e5a84bdff409bd05472 (DEFAULT_TABLE table=test-table [id=91d1dc9321684c87a9ba0cf0debd9154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:33.175118 26221 tablet_service.cc:1467] Processing CreateTablet for tablet cb72ce347abc4e5a84bdff409bd05472 (DEFAULT_TABLE table=test-table [id=91d1dc9321684c87a9ba0cf0debd9154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:33.175583 26383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb72ce347abc4e5a84bdff409bd05472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:33.175587 26221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb72ce347abc4e5a84bdff409bd05472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:33.179860 26566 tablet_bootstrap.cc:492] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: Bootstrap starting.
I20260812 02:43:33.180992 26567 tablet_bootstrap.cc:492] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: Bootstrap starting.
I20260812 02:43:33.182034 26566 tablet_bootstrap.cc:654] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:33.183007 26567 tablet_bootstrap.cc:654] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:33.184466 26566 tablet_bootstrap.cc:492] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: No bootstrap required, opened a new log
I20260812 02:43:33.184640 26566 ts_tablet_manager.cc:1397] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 02:43:33.185328 26567 tablet_bootstrap.cc:492] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: No bootstrap required, opened a new log
I20260812 02:43:33.185523 26567 ts_tablet_manager.cc:1397] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260812 02:43:33.185567 26566 raft_consensus.cc:348] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.185827 26566 raft_consensus.cc:374] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:33.185923 26566 raft_consensus.cc:729] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8a58be1415e4d268335e44dec2c1baf, State: Initialized, Role: FOLLOWER
I20260812 02:43:33.186213 26566 consensus_queue.cc:260] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.186306 26567 raft_consensus.cc:348] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.186596 26567 raft_consensus.cc:374] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:33.186697 26567 raft_consensus.cc:729] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1cd287a35546828e6a22f690827612, State: Initialized, Role: FOLLOWER
I20260812 02:43:33.186940 26567 consensus_queue.cc:260] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.188102 26566 ts_tablet_manager.cc:1428] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 02:43:33.189680 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.188605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.193:57343)
0812 02:43:33.189217 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.189389 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"threads_started":1}
I20260812 02:43:33.191077 26570 tablet_bootstrap.cc:492] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: Bootstrap starting.
I20260812 02:43:33.191628 26133 heartbeater.cc:503] Master 127.17.196.253:46567 was elected leader, sending a full tablet report...
I20260812 02:43:33.192502 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.190988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.193:39691)
0812 02:43:33.192143 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.192283 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":995}
I20260812 02:43:33.194095 26567 ts_tablet_manager.cc:1428] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: Time spent starting tablet: real 0.008s user 0.000s sys 0.007s
I20260812 02:43:33.195560 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.194405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.195:41495)
0812 02:43:33.195109 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.195231 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":395,"threads_started":1}
I20260812 02:43:33.196662 26460 heartbeater.cc:503] Master 127.17.196.253:46567 was elected leader, sending a full tablet report...
I20260812 02:43:33.196169 26570 tablet_bootstrap.cc:654] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:33.198726 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.196194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.195:57777)
0812 02:43:33.198363 (+ 2169us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.198510 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2056,"thread_start_us":1844,"threads_started":1}
I20260812 02:43:33.200603 26570 tablet_bootstrap.cc:492] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: No bootstrap required, opened a new log
I20260812 02:43:33.200788 26570 ts_tablet_manager.cc:1397] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260812 02:43:33.201659 26570 raft_consensus.cc:348] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.201882 26570 raft_consensus.cc:374] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:33.201944 26570 raft_consensus.cc:729] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af9696aa3b441cabaf65f148ff973f2, State: Initialized, Role: FOLLOWER
I20260812 02:43:33.202162 26570 consensus_queue.cc:260] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.202999 26570 ts_tablet_manager.cc:1428] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 02:43:33.205030 26296 heartbeater.cc:503] Master 127.17.196.253:46567 was elected leader, sending a full tablet report...
I20260812 02:43:33.205425 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.203314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.194:40123)
0812 02:43:33.204856 (+ 1542us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.205020 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1374,"thread_start_us":377,"threads_started":1}
I20260812 02:43:33.206712 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.206172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.194:59243)
0812 02:43:33.206421 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.206547 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 02:43:33.391222 26578 raft_consensus.cc:482] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:33.391539 26578 raft_consensus.cc:504] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.393580 26578 leader_election.cc:288] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8a58be1415e4d268335e44dec2c1baf (127.17.196.193:34391), 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795)
I20260812 02:43:33.402161 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.394150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:60895 (local address 127.17.196.193:34391)
0812 02:43:33.394339 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.394347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.394498 (+ 151us) server_negotiation.cc:407] Connection header received
0812 02:43:33.394921 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.394929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.395022 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.395144 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:33.396002 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.396916 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.400107 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.400987 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.401125 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.401262 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.401691 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.401841 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 02:43:33.402227 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.393886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:34391 (local address 127.17.196.194:60895)
0812 02:43:33.394357 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.394385 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:33.394674 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:33.395276 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:33.395281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:33.395323 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:33.395805 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.395829 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.397058 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.397067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.399901 (+ 2834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.399922 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.401254 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.401259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.401471 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.401541 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 02:43:33.401636 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 02:43:33.401761 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":151,"spinlock_wait_cycles":7424}
I20260812 02:43:33.403203 26075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb72ce347abc4e5a84bdff409bd05472" candidate_uuid: "4af9696aa3b441cabaf65f148ff973f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8a58be1415e4d268335e44dec2c1baf" is_pre_election: true
I20260812 02:43:33.403728 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.394185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:40517 (local address 127.17.196.195:43795)
0812 02:43:33.394448 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.394455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.394898 (+ 443us) server_negotiation.cc:407] Connection header received
0812 02:43:33.395022 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.395028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.395114 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.395237 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:33.396124 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.397073 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.401331 (+ 4258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.402625 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.402803 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.402897 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.403278 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.403381 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:43:33.403797 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.393886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:43795 (local address 127.17.196.194:40517)
0812 02:43:33.394765 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.394785 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:33.394902 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:33.395265 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:33.395272 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:33.395305 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:33.395884 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.395897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.397214 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.397223 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.399901 (+ 2678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:33.399922 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.402862 (+ 2940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.402870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:33.403104 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.403138 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:43:33.403248 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 02:43:33.403403 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":280,"thread_start_us":413,"threads_started":1}
I20260812 02:43:33.403712 26075 raft_consensus.cc:2454] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af9696aa3b441cabaf65f148ff973f2 in term 0.
I20260812 02:43:33.404417 26170 leader_election.cc:302] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af9696aa3b441cabaf65f148ff973f2, f8a58be1415e4d268335e44dec2c1baf; no voters:
I20260812 02:43:33.404568 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb72ce347abc4e5a84bdff409bd05472" candidate_uuid: "4af9696aa3b441cabaf65f148ff973f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b1cd287a35546828e6a22f690827612" is_pre_election: true
I20260812 02:43:33.404834 26404 raft_consensus.cc:2454] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af9696aa3b441cabaf65f148ff973f2 in term 0.
I20260812 02:43:33.404906 26578 raft_consensus.cc:2781] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:33.405077 26578 raft_consensus.cc:482] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:33.405208 26578 raft_consensus.cc:3037] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:33.407950 26578 raft_consensus.cc:504] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.408699 26578 leader_election.cc:288] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [CANDIDATE]: Term 1 election: Requested vote from peers f8a58be1415e4d268335e44dec2c1baf (127.17.196.193:34391), 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795)
I20260812 02:43:33.409240 26075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb72ce347abc4e5a84bdff409bd05472" candidate_uuid: "4af9696aa3b441cabaf65f148ff973f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8a58be1415e4d268335e44dec2c1baf"
I20260812 02:43:33.409349 26404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb72ce347abc4e5a84bdff409bd05472" candidate_uuid: "4af9696aa3b441cabaf65f148ff973f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b1cd287a35546828e6a22f690827612"
I20260812 02:43:33.409574 26404 raft_consensus.cc:3037] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:33.409528 26075 raft_consensus.cc:3037] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:33.412811 26404 raft_consensus.cc:2454] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af9696aa3b441cabaf65f148ff973f2 in term 1.
I20260812 02:43:33.412810 26075 raft_consensus.cc:2454] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af9696aa3b441cabaf65f148ff973f2 in term 1.
I20260812 02:43:33.413324 26168 leader_election.cc:302] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4af9696aa3b441cabaf65f148ff973f2, 8b1cd287a35546828e6a22f690827612, f8a58be1415e4d268335e44dec2c1baf; no voters:
I20260812 02:43:33.413746 26578 raft_consensus.cc:2781] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:33.414276 26578 raft_consensus.cc:686] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 LEADER]: Becoming Leader. State: Replica: 4af9696aa3b441cabaf65f148ff973f2, State: Running, Role: LEADER
I20260812 02:43:33.414722 26578 consensus_queue.cc:237] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } }
I20260812 02:43:33.415514 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.415040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.194:36091)
0812 02:43:33.415266 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.415369 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260812 02:43:33.420128 25695 catalog_manager.cc:5495] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "4af9696aa3b441cabaf65f148ff973f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af9696aa3b441cabaf65f148ff973f2" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 42369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 } health_report { overall_health: UNKNOWN } } }
I20260812 02:43:33.467154 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.461574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.17.196.194:42369)
0812 02:43:33.461795 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.461800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.461818 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:43:33.461952 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.461956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.462049 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.462218 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:33.462579 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.463343 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.464763 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.465192 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.465294 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.465945 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:33.466532 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:33.466668 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.466841 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.466919 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:43:33.472028 26404 raft_consensus.cc:1261] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 1 FOLLOWER]: Refusing update from remote peer 4af9696aa3b441cabaf65f148ff973f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:33.472445 26075 raft_consensus.cc:1261] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 1 FOLLOWER]: Refusing update from remote peer 4af9696aa3b441cabaf65f148ff973f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:33.472934 26581 consensus_queue.cc:1035] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b1cd287a35546828e6a22f690827612" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 43795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:33.473338 26578 consensus_queue.cc:1035] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8a58be1415e4d268335e44dec2c1baf" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 34391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:33.476275 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.475530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.195:56587)
0812 02:43:33.475835 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.475970 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260812 02:43:33.477056 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.476440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.193:48907)
0812 02:43:33.476652 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.476771 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 02:43:33.484789 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.484186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.193:37299)
0812 02:43:33.484490 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.484602 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260812 02:43:33.485857 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.485306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.195:58025)
0812 02:43:33.485528 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.485659 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 02:43:33.486199 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.485259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.17.196.194:38757)
0812 02:43:33.485837 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.485964 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418}
I20260812 02:43:33.500710 25696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:33.501042 25696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:33.508230 25696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1f1396065e740869a40a86ff99bf02f: Sending DeleteTablet for 3 replicas of tablet cb72ce347abc4e5a84bdff409bd05472
I20260812 02:43:33.509402 26052 tablet_service.cc:1514] Processing DeleteTablet for tablet cb72ce347abc4e5a84bdff409bd05472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:33 UTC) from {username='oryx', principal='oryx/127.17.196.253@KRBTEST.COM'} at 127.0.0.1:55342
I20260812 02:43:33.509707 26221 tablet_service.cc:1514] Processing DeleteTablet for tablet cb72ce347abc4e5a84bdff409bd05472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:33 UTC) from {username='oryx', principal='oryx/127.17.196.253@KRBTEST.COM'} at 127.0.0.1:35834
I20260812 02:43:33.509688 26383 tablet_service.cc:1514] Processing DeleteTablet for tablet cb72ce347abc4e5a84bdff409bd05472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:33 UTC) from {username='oryx', principal='oryx/127.17.196.253@KRBTEST.COM'} at 127.0.0.1:45266
I20260812 02:43:33.515159 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 25883
I20260812 02:43:33.520190 26598 tablet_replica.cc:330] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: stopping tablet replica
I20260812 02:43:33.520491 26598 raft_consensus.cc:2229] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:43:33.521278 26598 raft_consensus.cc:2258] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:33.522193 26599 tablet_replica.cc:330] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: stopping tablet replica
I20260812 02:43:33.522449 26599 raft_consensus.cc:2229] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:33.522872 26599 raft_consensus.cc:2258] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:33.524415 26598 ts_tablet_manager.cc:1905] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:33.525756 26596 tablet_replica.cc:330] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: stopping tablet replica
I20260812 02:43:33.525976 26596 raft_consensus.cc:2229] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:33.526328 26596 raft_consensus.cc:2258] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:33.527654 26596 ts_tablet_manager.cc:1905] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:33.533365 26598 ts_tablet_manager.cc:1918] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:33.534209 26599 ts_tablet_manager.cc:1905] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:33.534638 26596 ts_tablet_manager.cc:1918] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:33.534801 26596 log.cc:1196] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/cb72ce347abc4e5a84bdff409bd05472
I20260812 02:43:33.535336 26596 ts_tablet_manager.cc:1939] T cb72ce347abc4e5a84bdff409bd05472 P f8a58be1415e4d268335e44dec2c1baf: Deleting consensus metadata
I20260812 02:43:33.536664 25682 catalog_manager.cc:4841] TS f8a58be1415e4d268335e44dec2c1baf (127.17.196.193:34391): tablet cb72ce347abc4e5a84bdff409bd05472 (table test-table [id=91d1dc9321684c87a9ba0cf0debd9154]) successfully deleted
I20260812 02:43:33.541365 26599 ts_tablet_manager.cc:1918] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:33.541576 26599 log.cc:1196] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/cb72ce347abc4e5a84bdff409bd05472
I20260812 02:43:33.541812 26598 log.cc:1196] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/cb72ce347abc4e5a84bdff409bd05472
I20260812 02:43:33.542080 26599 ts_tablet_manager.cc:1939] T cb72ce347abc4e5a84bdff409bd05472 P 8b1cd287a35546828e6a22f690827612: Deleting consensus metadata
I20260812 02:43:33.542550 26598 ts_tablet_manager.cc:1939] T cb72ce347abc4e5a84bdff409bd05472 P 4af9696aa3b441cabaf65f148ff973f2: Deleting consensus metadata
I20260812 02:43:33.543473 25680 catalog_manager.cc:4841] TS 8b1cd287a35546828e6a22f690827612 (127.17.196.195:43795): tablet cb72ce347abc4e5a84bdff409bd05472 (table test-table [id=91d1dc9321684c87a9ba0cf0debd9154]) successfully deleted
I20260812 02:43:33.543865 25681 catalog_manager.cc:4841] TS 4af9696aa3b441cabaf65f148ff973f2 (127.17.196.194:42369): tablet cb72ce347abc4e5a84bdff409bd05472 (table test-table [id=91d1dc9321684c87a9ba0cf0debd9154]) successfully deleted
I20260812 02:43:33.549672 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.516474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.17.196.193:34391)
0812 02:43:33.517129 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.517139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.531200 (+ 14061us) server_negotiation.cc:407] Connection header received
0812 02:43:33.531719 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.531725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.531837 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.531923 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:33.534264 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.535494 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.537695 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.538326 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.538453 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.542422 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:33.542444 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:33.542465 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:33.542516 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:33.546417 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:33.547235 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:33.547243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:33.547246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:33.547391 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:33.547848 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:33.547859 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:33.547861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:33.548311 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:33.548639 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.549008 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.549271 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471}
I20260812 02:43:33.550347 26121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:33.602835 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.602300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41756)
0812 02:43:33.602519 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.602672 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 02:43:33.604024 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.603653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34567 (local address 127.0.0.1:41760)
0812 02:43:33.603810 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:33.603902 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:34567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 02:43:33.759598 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 25883
I20260812 02:43:33.789309 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 26146
I20260812 02:43:33.805636 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.790379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35844 (local address 127.17.196.194:42369)
0812 02:43:33.790741 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.790747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.790769 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:43:33.790966 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.790973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.791061 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.791185 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:33.792697 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.793819 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.795207 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.795667 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.795817 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.798748 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:33.798760 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:33.798776 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:33.798832 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:33.802474 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:33.803255 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:33.803269 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:33.803272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:33.803424 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:33.803832 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:33.803837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:33.803841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:33.804223 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:33.804569 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.804874 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.805105 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260812 02:43:33.806725 26289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:33.936008 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 26146
I20260812 02:43:33.960342 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 26310
I20260812 02:43:33.975867 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:33.961307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45280 (local address 127.17.196.195:43795)
0812 02:43:33.961913 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:33.961917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:33.961932 (+ 15us) server_negotiation.cc:407] Connection header received
0812 02:43:33.961989 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:33.961993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:33.962058 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:33.962250 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:33.963792 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.964555 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.966157 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:33.966571 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:33.966728 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:33.969703 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:33.969725 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:33.969744 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:33.969784 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:33.972825 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:33.973639 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:33.973650 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:33.973655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:33.973802 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:33.974212 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:33.974218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:33.974221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:33.974615 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:33.974953 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:33.975267 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 02:43:33.975515 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":353,"threads_started":1}
I20260812 02:43:33.976747 26452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:34.077032 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 26310
I20260812 02:43:34.096904 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 25538
I20260812 02:43:34.098197 25730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:34.226533 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 25538
I20260812 02:43:34.254456 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 25655
I20260812 02:43:34.255755 25954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:34.376767 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 25655
2026-08-12T02:43:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11212 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:43:34 dist-test-slave-ss4n krb5kdc[26611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:43:34 dist-test-slave-ss4n krb5kdc[26611](info): set up 2 sockets
Aug 12 02:43:34 dist-test-slave-ss4n krb5kdc[26611](info): commencing operation
krb5kdc: starting...
W20260812 02:43:35.649405 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:43:35 dist-test-slave-ss4n krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:43:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.254.keytab.
Entry for principal oryx/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.254.keytab.
I20260812 02:43:35.951046 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37945
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.254.keytab
--principal=oryx/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:37945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:36.305383 26628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:36.305830 26628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:36.305998 26628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:36.313856 26628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:36.314016 26628 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 02:43:36.314110 26628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:36.314234 26628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:36.314324 26628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:36.314414 26628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:36.325688 26628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:37945
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.254.keytab
--principal=oryx/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:36.327333 26628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:36.329116 26628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:36.342892 26633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:36.343132 26634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:36.343349 26628 server_base.cc:1034] running on GCE node
W20260812 02:43:36.342942 26636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:36.344532 26628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:36.346271 26628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:36.347551 26628 hybrid_clock.cc:648] HybridClock initialized: now 1786502616347479 us; error 47 us; skew 500 ppm
I20260812 02:43:36.347970 26628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:36 dist-test-slave-ss4n krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502616, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:36.356489 26628 init.cc:377] Logged in from keytab as oryx/127.17.196.254@KRBTEST.COM (short username oryx)
I20260812 02:43:36.358357 26628 webserver.cc:460] Webserver started at http://127.17.196.254:44471/ using document root <none> and password file <none>
I20260812 02:43:36.359133 26628 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:36.359234 26628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:36.359524 26628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:36.362850 26628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "1670d18175594aa2bf3a651933ef3ec0"
format_stamp: "Formatted at 2026-08-12 02:43:36 on dist-test-slave-ss4n"
server_key: "634f20be87d0fb1a5760285857751117"
server_key_iv: "7a522c2bc5a6ddf9d8c8cb208a6dbab2"
server_key_version: "encryptionkey@0"
I20260812 02:43:36.363719 26628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "1670d18175594aa2bf3a651933ef3ec0"
format_stamp: "Formatted at 2026-08-12 02:43:36 on dist-test-slave-ss4n"
server_key: "634f20be87d0fb1a5760285857751117"
server_key_iv: "7a522c2bc5a6ddf9d8c8cb208a6dbab2"
server_key_version: "encryptionkey@0"
I20260812 02:43:36.369719 26628 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:43:36.373859 26644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:36.375533 26628 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:43:36.375782 26628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "1670d18175594aa2bf3a651933ef3ec0"
format_stamp: "Formatted at 2026-08-12 02:43:36 on dist-test-slave-ss4n"
server_key: "634f20be87d0fb1a5760285857751117"
server_key_iv: "7a522c2bc5a6ddf9d8c8cb208a6dbab2"
server_key_version: "encryptionkey@0"
I20260812 02:43:36.376060 26628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:36.395404 26628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:36.400046 26628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:36.400379 26628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:36.425402 26628 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:37945
I20260812 02:43:36.425427 26710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:37945 every 8 connection(s)
I20260812 02:43:36.426961 26628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:36.431038 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 26628
I20260812 02:43:36.431273 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:36.432135 18195 external_mini_cluster.cc:1428] Setting key 49650a94adfad1307d4a02727d5f3b3d
I20260812 02:43:36.432221 26711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:36.445808 26711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: Bootstrap starting.
Aug 12 02:43:36 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:36.449622 26711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:36.450938 26711 log.cc:824] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:36.454661 26711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: No bootstrap required, opened a new log
I20260812 02:43:36.457451 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:36.434317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46504 (local address 127.17.196.254:37945)
0812 02:43:36.435159 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:36.435175 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:36.435221 (+ 46us) server_negotiation.cc:407] Connection header received
0812 02:43:36.437320 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:36.437345 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:36.437895 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:36.438371 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:36.440018 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:36.441321 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:36.442713 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:36.443503 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:36.443778 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:36.448418 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:36.448428 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:36.448444 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:36.448485 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:36.452337 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:36.453160 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:36.453168 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:36.453176 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:36.453416 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:36.453952 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:36.453957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:36.453961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:36.454722 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:36.455076 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:36.455527 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 02:43:36.455992 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":382,"threads_started":1}
I20260812 02:43:36.464043 26711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1670d18175594aa2bf3a651933ef3ec0" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37945 } }
I20260812 02:43:36.464440 26711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:36.464571 26711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1670d18175594aa2bf3a651933ef3ec0, State: Initialized, Role: FOLLOWER
I20260812 02:43:36.465178 26711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1670d18175594aa2bf3a651933ef3ec0" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37945 } }
I20260812 02:43:36.465420 26711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:36.465597 26711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:36.465793 26711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:36.468158 26711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1670d18175594aa2bf3a651933ef3ec0" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37945 } }
I20260812 02:43:36.468756 26711 leader_election.cc:302] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1670d18175594aa2bf3a651933ef3ec0; no voters:
I20260812 02:43:36.469414 26711 leader_election.cc:288] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:36.469774 26716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:36.470561 26716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [term 1 LEADER]: Becoming Leader. State: Replica: 1670d18175594aa2bf3a651933ef3ec0, State: Running, Role: LEADER
I20260812 02:43:36.471194 26716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1670d18175594aa2bf3a651933ef3ec0" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37945 } }
I20260812 02:43:36.471906 26711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:36.474993 26717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1670d18175594aa2bf3a651933ef3ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1670d18175594aa2bf3a651933ef3ec0" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37945 } } }
I20260812 02:43:36.475469 26717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:36.474969 26718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1670d18175594aa2bf3a651933ef3ec0. Latest consensus state: current_term: 1 leader_uuid: "1670d18175594aa2bf3a651933ef3ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1670d18175594aa2bf3a651933ef3ec0" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37945 } } }
I20260812 02:43:36.475869 26718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0 [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:36.479010 26724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:36.486340 26724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:36.497753 26724 catalog_manager.cc:1347] Generated new cluster ID: b884cd21cb56458984944f3f01c4a961
I20260812 02:43:36.497921 26724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:36.526496 26724 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:36.528826 26724 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:36.542869 26724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: Generated new TSK 0
I20260812 02:43:36.543860 26724 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab.
Entry for principal oryx/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab.
I20260812 02:43:36.626652 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37945
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab
--principal=oryx/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:36.999395 26740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:36.999763 26740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:36.999936 26740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:37.007649 26740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:37.007833 26740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:37.007977 26740 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:43:37.008114 26740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:37.018710 26740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab
--principal=oryx/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:37.020180 26740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:37.021713 26740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:37.036684 26746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:37.036680 26749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:37.036803 26751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:37.038005 26740 server_base.cc:1034] running on GCE node
I20260812 02:43:37.038703 26740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:37.039702 26740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:37.041046 26740 hybrid_clock.cc:648] HybridClock initialized: now 1786502617040941 us; error 66 us; skew 500 ppm
I20260812 02:43:37.041445 26740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:37 dist-test-slave-ss4n krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502617, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:37.045396 26740 init.cc:377] Logged in from keytab as oryx/127.17.196.193@KRBTEST.COM (short username oryx)
I20260812 02:43:37.047372 26740 webserver.cc:460] Webserver started at http://127.17.196.193:46387/ using document root <none> and password file <none>
I20260812 02:43:37.048228 26740 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:37.048385 26740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:37.048770 26740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:37.052265 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "135224b9aab54bb18cc9319160516df3"
format_stamp: "Formatted at 2026-08-12 02:43:37 on dist-test-slave-ss4n"
server_key: "414018ddbf06b1dc8fd9e8965da052b1"
server_key_iv: "aba0f35efd727611be82c08449e0264e"
server_key_version: "encryptionkey@0"
I20260812 02:43:37.053241 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "135224b9aab54bb18cc9319160516df3"
format_stamp: "Formatted at 2026-08-12 02:43:37 on dist-test-slave-ss4n"
server_key: "414018ddbf06b1dc8fd9e8965da052b1"
server_key_iv: "aba0f35efd727611be82c08449e0264e"
server_key_version: "encryptionkey@0"
I20260812 02:43:37.059624 26740 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260812 02:43:37.064145 26758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:37.066080 26740 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:43:37.066306 26740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "135224b9aab54bb18cc9319160516df3"
format_stamp: "Formatted at 2026-08-12 02:43:37 on dist-test-slave-ss4n"
server_key: "414018ddbf06b1dc8fd9e8965da052b1"
server_key_iv: "aba0f35efd727611be82c08449e0264e"
server_key_version: "encryptionkey@0"
I20260812 02:43:37.066540 26740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:37.089850 26740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:37.094875 26740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:37.095146 26740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:37.098342 26740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:37.098456 26740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:37.098588 26740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:37.098706 26740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:43:37 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502617, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:37.124413 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:37.098976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:54019 (local address 127.17.196.254:37945)
0812 02:43:37.099689 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:37.099694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:37.100972 (+ 1278us) server_negotiation.cc:407] Connection header received
0812 02:43:37.103479 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:37.103486 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:37.103616 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:37.103770 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:37.106644 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.107821 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.110405 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.111050 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.111218 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:37.116997 (+ 5779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:37.117003 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:37.117006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:37.117050 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:37.120911 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:37.121898 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:37.121907 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:37.121911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:37.122056 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:37.122564 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:37.122574 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:37.122577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:37.122858 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:37.123036 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:37.123814 (+ 778us) server_negotiation.cc:299] Negotiation successful
0812 02:43:37.124045 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":492,"threads_started":1}
I20260812 02:43:37.125851 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:37.099691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37945 (local address 127.17.196.193:54019)
0812 02:43:37.100741 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:37.100798 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:37.103120 (+ 2322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:37.104135 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:37.104156 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:37.105145 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:37.106295 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:37.106340 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.108028 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.108041 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:37.110143 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:37.110164 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.111373 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.111381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:37.111680 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:37.112671 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:37.112716 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:37.116673 (+ 3957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:37.121127 (+ 4454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:37.121138 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:37.121162 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:37.121735 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:37.122192 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:37.122199 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:37.122203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:37.122424 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:37.123011 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:37.123020 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:37.123416 (+ 396us) client_negotiation.cc:769] Sending connection context
0812 02:43:37.123704 (+ 288us) client_negotiation.cc:240] Negotiation successful
0812 02:43:37.124268 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":424,"threads_started":1}
I20260812 02:43:37.149597 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:37.132873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:34787 (local address 127.17.196.254:37945)
0812 02:43:37.133172 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:37.133182 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:37.133874 (+ 692us) server_negotiation.cc:407] Connection header received
0812 02:43:37.134129 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:37.134138 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:37.134258 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:37.134410 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:37.135417 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.136960 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.139146 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.139965 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.140154 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:37.142909 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:37.142914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:37.142917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:37.142952 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:37.146377 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:37.147408 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:37.147414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:37.147417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:37.147557 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:37.147994 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:37.147998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:37.148000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:37.148315 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:37.148504 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:37.148937 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 02:43:37.149186 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:43:37.149720 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:37.132899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37945 (local address 127.17.196.193:34787)
0812 02:43:37.133725 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:37.133746 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:37.133941 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:37.134437 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:37.134440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:37.134923 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:37.135247 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:37.135258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.137168 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.137176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:37.138922 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:37.138947 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.140226 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.140232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:37.140349 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:37.141157 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:37.141192 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:37.142621 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:37.146530 (+ 3909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:37.146539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:37.146543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:37.147262 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:37.147694 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:37.147699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:37.147703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:37.147855 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:37.148443 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:37.148447 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:37.148654 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 02:43:37.148800 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 02:43:37.149179 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":411,"threads_started":1}
I20260812 02:43:37.158950 26740 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:39641
I20260812 02:43:37.159044 26890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:39641 every 8 connection(s)
I20260812 02:43:37.160337 26740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:37.163053 26891 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37945
I20260812 02:43:37.163352 26891 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:37.164093 26891 heartbeater.cc:511] Master 127.17.196.254:37945 requested a full tablet report, sending...
I20260812 02:43:37.166463 26664 ts_manager.cc:194] Registered new tserver with Master: 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641)
I20260812 02:43:37.168090 26664 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:54019
I20260812 02:43:37.168603 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 26740
I20260812 02:43:37.168810 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:43:37.169648 18195 external_mini_cluster.cc:1428] Setting key 6b6a32f7952c9bf6a5f3c2bc778a789b
WARNING: no policy specified for oryx/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab.
Entry for principal oryx/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab.
I20260812 02:43:37.234622 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37945
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab
--principal=oryx/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:37.600888 26895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:37.601226 26895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:37.601404 26895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:37.609289 26895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:37.609428 26895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:37.609642 26895 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:43:37.609776 26895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:37.620363 26895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab
--principal=oryx/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:37.621903 26895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:37.623588 26895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:37.638682 26903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:37.638688 26902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:37.639307 26905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:37.639696 26895 server_base.cc:1034] running on GCE node
I20260812 02:43:37.640424 26895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:37.641600 26895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:37.642865 26895 hybrid_clock.cc:648] HybridClock initialized: now 1786502617642798 us; error 43 us; skew 500 ppm
I20260812 02:43:37.643221 26895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:37 dist-test-slave-ss4n krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502617, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:37.646716 26895 init.cc:377] Logged in from keytab as oryx/127.17.196.194@KRBTEST.COM (short username oryx)
I20260812 02:43:37.648350 26895 webserver.cc:460] Webserver started at http://127.17.196.194:39911/ using document root <none> and password file <none>
I20260812 02:43:37.649163 26895 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:37.649309 26895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:37.649736 26895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:37.652881 26895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "1e7b87ffb31343aaa8a2802f1b869d68"
format_stamp: "Formatted at 2026-08-12 02:43:37 on dist-test-slave-ss4n"
server_key: "58d78d3714cbe7009c32c6e037f71aee"
server_key_iv: "37ae6f06af72181d0fc0c4d745c8045a"
server_key_version: "encryptionkey@0"
I20260812 02:43:37.653890 26895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "1e7b87ffb31343aaa8a2802f1b869d68"
format_stamp: "Formatted at 2026-08-12 02:43:37 on dist-test-slave-ss4n"
server_key: "58d78d3714cbe7009c32c6e037f71aee"
server_key_iv: "37ae6f06af72181d0fc0c4d745c8045a"
server_key_version: "encryptionkey@0"
I20260812 02:43:37.659677 26895 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:43:37.664363 26913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:37.666169 26895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:43:37.666389 26895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "1e7b87ffb31343aaa8a2802f1b869d68"
format_stamp: "Formatted at 2026-08-12 02:43:37 on dist-test-slave-ss4n"
server_key: "58d78d3714cbe7009c32c6e037f71aee"
server_key_iv: "37ae6f06af72181d0fc0c4d745c8045a"
server_key_version: "encryptionkey@0"
I20260812 02:43:37.666648 26895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:37.695453 26895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:37.700217 26895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:37.700543 26895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:37.703754 26895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:37.703868 26895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:37.704020 26895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:37.704107 26895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:43:37 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502617, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:37.731451 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:37.704253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:47949 (local address 127.17.196.254:37945)
0812 02:43:37.705189 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:37.705195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:37.706282 (+ 1087us) server_negotiation.cc:407] Connection header received
0812 02:43:37.708890 (+ 2608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:37.708896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:37.709001 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:37.709153 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:37.711766 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.712878 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.715801 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.716506 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.716669 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:37.722679 (+ 6010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:37.722691 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:37.722699 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:37.722764 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:37.727237 (+ 4473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:37.728379 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:37.728385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:37.728389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:37.728607 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:37.729136 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:37.729141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:37.729145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:37.729456 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:37.729673 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:37.730933 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0812 02:43:37.731161 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":720,"threads_started":1}
I20260812 02:43:37.733567 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:37.705032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37945 (local address 127.17.196.194:47949)
0812 02:43:37.706074 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:37.706126 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:37.708388 (+ 2262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:37.709474 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:37.709519 (+ 45us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:37.710479 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:37.711526 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:37.711550 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.713071 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.713075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:37.715583 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:37.715599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.716789 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.716796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:37.717058 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:37.717972 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:37.718016 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:37.722255 (+ 4239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:37.727408 (+ 5153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:37.727424 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:37.727448 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:37.728211 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:37.728735 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:37.728740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:37.728744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:37.728996 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:37.729690 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:37.729703 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:37.730342 (+ 639us) client_negotiation.cc:769] Sending connection context
0812 02:43:37.730777 (+ 435us) client_negotiation.cc:240] Negotiation successful
0812 02:43:37.731545 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":504,"threads_started":1}
I20260812 02:43:37.759030 26895 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:46567
I20260812 02:43:37.759050 27056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:46567 every 8 connection(s)
I20260812 02:43:37.760776 26895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:37.761541 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:37.742791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:46475 (local address 127.17.196.254:37945)
0812 02:43:37.743062 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:37.743072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:37.746339 (+ 3267us) server_negotiation.cc:407] Connection header received
0812 02:43:37.746575 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:37.746585 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:37.746706 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:37.746844 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:37.747917 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.749043 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.751417 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.751999 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.752197 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:37.755232 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:37.755237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:37.755241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:37.755276 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:37.758488 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:37.759348 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:37.759352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:37.759354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:37.759476 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:37.759922 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:37.759925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:37.759927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:37.760250 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:37.760378 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:37.760969 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 02:43:37.761168 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:43:37.761855 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:37.742125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37945 (local address 127.17.196.194:46475)
0812 02:43:37.746127 (+ 4002us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:37.746155 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:37.746384 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:37.746899 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:37.746904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:37.747406 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:37.747731 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:37.747742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.749270 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.749279 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:37.751215 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:37.751232 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:37.752507 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:37.752512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:37.752688 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:37.753608 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:37.753639 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:37.754924 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:37.758636 (+ 3712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:37.758643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:37.758647 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:37.759184 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:37.759605 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:37.759611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:37.759615 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:37.759796 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:37.760380 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:37.760385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:37.760606 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 02:43:37.760799 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 02:43:37.761176 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3801,"thread_start_us":3628,"threads_started":1}
I20260812 02:43:37.763688 27057 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37945
I20260812 02:43:37.763950 27057 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:37.764814 27057 heartbeater.cc:511] Master 127.17.196.254:37945 requested a full tablet report, sending...
I20260812 02:43:37.765872 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 26895
I20260812 02:43:37.766069 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:43:37.766520 18195 external_mini_cluster.cc:1428] Setting key 72fda71d3ee1cd2ab618ecca1ddd30c4
I20260812 02:43:37.767198 26663 ts_manager.cc:194] Registered new tserver with Master: 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567)
I20260812 02:43:37.768554 26663 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:47949
WARNING: no policy specified for oryx/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab.
Entry for principal oryx/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab.
I20260812 02:43:37.834077 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37945
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab
--principal=oryx/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:43:38.171994 26891 heartbeater.cc:503] Master 127.17.196.254:37945 was elected leader, sending a full tablet report...
W20260812 02:43:38.204638 27062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:38.205014 27062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:38.205201 27062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:38.212855 27062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:38.213019 27062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:38.213172 27062 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:43:38.213310 27062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:38.223855 27062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab
--principal=oryx/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.196.254:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:38.225294 27062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:38.226889 27062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:38.241672 27070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:38.241734 27068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:38.241925 27067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:38.243314 27062 server_base.cc:1034] running on GCE node
I20260812 02:43:38.244179 27062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:38.245316 27062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:38.246675 27062 hybrid_clock.cc:648] HybridClock initialized: now 1786502618246618 us; error 55 us; skew 500 ppm
I20260812 02:43:38.247030 27062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:38 dist-test-slave-ss4n krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502618, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:38.250833 27062 init.cc:377] Logged in from keytab as oryx/127.17.196.195@KRBTEST.COM (short username oryx)
I20260812 02:43:38.252556 27062 webserver.cc:460] Webserver started at http://127.17.196.195:40727/ using document root <none> and password file <none>
I20260812 02:43:38.253412 27062 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:38.253583 27062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:38.253996 27062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:38.257107 27062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1"
format_stamp: "Formatted at 2026-08-12 02:43:38 on dist-test-slave-ss4n"
server_key: "f8d416403c34ff91b0c747a3c8775e70"
server_key_iv: "a620eac167414488f2cc189f38982b24"
server_key_version: "encryptionkey@0"
I20260812 02:43:38.258020 27062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1"
format_stamp: "Formatted at 2026-08-12 02:43:38 on dist-test-slave-ss4n"
server_key: "f8d416403c34ff91b0c747a3c8775e70"
server_key_iv: "a620eac167414488f2cc189f38982b24"
server_key_version: "encryptionkey@0"
I20260812 02:43:38.263670 27062 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:43:38.267562 27078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:38.269148 27062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:43:38.269343 27062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1"
format_stamp: "Formatted at 2026-08-12 02:43:38 on dist-test-slave-ss4n"
server_key: "f8d416403c34ff91b0c747a3c8775e70"
server_key_iv: "a620eac167414488f2cc189f38982b24"
server_key_version: "encryptionkey@0"
I20260812 02:43:38.269716 27062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:38.301714 27062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:38.306538 27062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:38.306876 27062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:38.309885 27062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:38.310011 27062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:38.310182 27062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:38.310281 27062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:43:38 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502618, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:38.338755 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.310843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:38911 (local address 127.17.196.254:37945)
0812 02:43:38.311400 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.311407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.312660 (+ 1253us) server_negotiation.cc:407] Connection header received
0812 02:43:38.315196 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.315207 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.315333 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.315474 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:38.318179 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.319338 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.322131 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.322801 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.322953 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.329560 (+ 6607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:38.329572 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:38.329576 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:38.329622 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:38.333802 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:38.335165 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:38.335176 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:38.335182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:38.335424 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:38.336222 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:38.336231 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:38.336236 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:38.336740 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:38.336968 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.338089 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.338346 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":275,"threads_started":1}
I20260812 02:43:38.339913 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.311524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37945 (local address 127.17.196.195:38911)
0812 02:43:38.312451 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:38.312512 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:38.314731 (+ 2219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:38.315795 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:38.315813 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:38.316754 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:38.317826 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.317863 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.319556 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.319562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.321942 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.321958 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.323160 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.323170 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.323385 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.324277 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:38.324324 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:38.329035 (+ 4711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:38.333996 (+ 4961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:38.334020 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:38.334048 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:38.334923 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:38.335622 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:38.335626 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:38.335628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:38.336074 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:38.336956 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:38.336968 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:38.337405 (+ 437us) client_negotiation.cc:769] Sending connection context
0812 02:43:38.337950 (+ 545us) client_negotiation.cc:240] Negotiation successful
0812 02:43:38.338571 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":426,"threads_started":1}
I20260812 02:43:38.364116 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.346419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:52685 (local address 127.17.196.254:37945)
0812 02:43:38.346664 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.346673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.347472 (+ 799us) server_negotiation.cc:407] Connection header received
0812 02:43:38.347718 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.347724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.347816 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.347942 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:38.349020 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.349901 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.352707 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.353323 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.353513 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.356648 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:38.356654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:38.356658 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:38.356697 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:38.360891 (+ 4194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:38.361829 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:38.361835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:38.361839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:38.361993 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:38.362413 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:38.362419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:38.362422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:38.362876 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:38.363055 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.363489 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.363736 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:43:38.364221 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.346410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37945 (local address 127.17.196.195:52685)
0812 02:43:38.347289 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:38.347313 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:38.347557 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:38.347981 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:38.347985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:38.348536 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:38.348840 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.348858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.350080 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.350088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.352500 (+ 2412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.352524 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.353613 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.353622 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.353913 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.354751 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:38.354784 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:38.356303 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:38.361099 (+ 4796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:38.361110 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:38.361113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:38.361681 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:38.362126 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:38.362132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:38.362146 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:38.362278 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:38.363025 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:38.363029 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:38.363227 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 02:43:38.363380 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 02:43:38.363747 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":458,"threads_started":1}
I20260812 02:43:38.378934 27062 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:34113
I20260812 02:43:38.378976 27219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:34113 every 8 connection(s)
I20260812 02:43:38.380474 27062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:38.383347 27220 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37945
I20260812 02:43:38.383626 27220 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:38.384485 27220 heartbeater.cc:511] Master 127.17.196.254:37945 requested a full tablet report, sending...
I20260812 02:43:38.386091 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 27062
I20260812 02:43:38.386238 26663 ts_manager.cc:194] Registered new tserver with Master: 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113)
I20260812 02:43:38.386281 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:43:38.386806 18195 external_mini_cluster.cc:1428] Setting key d2fe3c6a161ed5bb9aed6d89e25d745a
I20260812 02:43:38.387166 26663 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:38911
I20260812 02:43:38.390358 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:43:38 dist-test-slave-ss4n krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502618, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:43:38 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:38 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:43:38.460112 27087 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:38.448024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46510 (local address 127.17.196.254:37945)
0812 02:43:38.448378 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.448386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.448543 (+ 157us) server_negotiation.cc:407] Connection header received
0812 02:43:38.448712 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.448717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.448812 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.448908 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:38.450519 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.451308 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.452638 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.453042 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.453158 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.459838 (+ 6680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46510: BlockingRecv error: recv got EOF from 127.0.0.1:46510 (error 108)
Metrics: {"server-negotiator.queue_time_us":194}
Aug 12 02:43:38 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502618, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:38.481407 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.464702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46522 (local address 127.17.196.254:37945)
0812 02:43:38.464985 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.464992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.465248 (+ 256us) server_negotiation.cc:407] Connection header received
0812 02:43:38.465530 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.465536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.465629 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.465763 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:38.467094 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.467909 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.469385 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.470069 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.470238 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.475378 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:38.475384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:38.475388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:38.475432 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:38.478449 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:38.479194 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:38.479200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:38.479203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:38.479350 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:38.479853 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:38.479858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:38.479862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:38.480166 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:38.480443 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.480816 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.481068 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 02:43:38.486083 26663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:43:38.493325 26663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:38.521628 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.507392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:34113 (local address 127.0.0.1:53436)
0812 02:43:38.509045 (+ 1653us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:38.509109 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:38.509279 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:38.513109 (+ 3830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:38.513121 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:38.513165 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:38.513767 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.513783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.516164 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.516170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.518325 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.518339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.520548 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.520554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.520763 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.520818 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 02:43:38.520967 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 02:43:38.521174 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1378,"thread_start_us":1219,"threads_started":1}
I20260812 02:43:38.521644 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.507737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.17.196.195:34113)
0812 02:43:38.512267 (+ 4530us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.512278 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.512314 (+ 36us) server_negotiation.cc:407] Connection header received
0812 02:43:38.512417 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.512426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.512780 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.513974 (+ 1194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:38.514377 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.516000 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.518523 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.519980 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.520210 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.520403 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.521049 (+ 646us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.521247 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4295,"thread_start_us":4095,"threads_started":1}
I20260812 02:43:38.522543 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.510800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.17.196.193:39641)
0812 02:43:38.513210 (+ 2410us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.513222 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.513934 (+ 712us) server_negotiation.cc:407] Connection header received
0812 02:43:38.514259 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.514266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.514456 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.514634 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:38.515733 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.516830 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.519724 (+ 2894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.520811 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.520960 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.521141 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.522064 (+ 923us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.522269 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2213,"thread_start_us":1420,"threads_started":1}
I20260812 02:43:38.522722 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.507458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:46567 (local address 127.0.0.1:59658)
0812 02:43:38.509774 (+ 2316us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:38.509798 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:38.509910 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:38.514613 (+ 4703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:38.514618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:38.514645 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:38.515226 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.515238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.517137 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.517148 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.519947 (+ 2799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.519962 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.522014 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.522020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.522222 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.522246 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:43:38.522309 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 02:43:38.522399 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2188,"mutex_wait_us":32,"thread_start_us":2008,"threads_started":1}
I20260812 02:43:38.523314 27242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.510394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39641 (local address 127.0.0.1:55158)
0812 02:43:38.513814 (+ 3420us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:38.513838 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:38.514039 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:38.514868 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:38.514872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:38.514893 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:38.515540 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.515554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.516994 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.517000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.519523 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.519545 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.521359 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.521368 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.521571 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.521605 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:43:38.521708 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 02:43:38.523050 (+ 1342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3297,"mutex_wait_us":257,"thread_start_us":3082,"threads_started":1}
I20260812 02:43:38.523937 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.508051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59658 (local address 127.17.196.194:46567)
0812 02:43:38.513983 (+ 5932us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.513989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.514011 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:43:38.514094 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.514134 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.514362 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.515424 (+ 1062us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:38.515657 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.516993 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.520134 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.521528 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.522909 (+ 1381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.523133 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.523213 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.523385 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5733,"thread_start_us":5568,"threads_started":1}
I20260812 02:43:38.526252 27135 tablet_service.cc:1467] Processing CreateTablet for tablet 967bb861839a4d2aabc2d4768ff1ca3f (DEFAULT_TABLE table=test-table [id=1760f5bb1d1b47d38575fd3bcac536cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:38.527380 26812 tablet_service.cc:1467] Processing CreateTablet for tablet 967bb861839a4d2aabc2d4768ff1ca3f (DEFAULT_TABLE table=test-table [id=1760f5bb1d1b47d38575fd3bcac536cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:38.527942 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967bb861839a4d2aabc2d4768ff1ca3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.527850 26973 tablet_service.cc:1467] Processing CreateTablet for tablet 967bb861839a4d2aabc2d4768ff1ca3f (DEFAULT_TABLE table=test-table [id=1760f5bb1d1b47d38575fd3bcac536cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:38.528662 26812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967bb861839a4d2aabc2d4768ff1ca3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.529237 26973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967bb861839a4d2aabc2d4768ff1ca3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.541607 27244 tablet_bootstrap.cc:492] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: Bootstrap starting.
I20260812 02:43:38.543924 27245 tablet_bootstrap.cc:492] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Bootstrap starting.
I20260812 02:43:38.545775 27246 tablet_bootstrap.cc:492] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: Bootstrap starting.
I20260812 02:43:38.546214 27244 tablet_bootstrap.cc:654] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:38.547705 27245 tablet_bootstrap.cc:654] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:38.548722 27244 log.cc:824] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:38.548995 27246 tablet_bootstrap.cc:654] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:38.549041 27245 log.cc:824] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:38.550309 27246 log.cc:824] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:38.552778 27245 tablet_bootstrap.cc:492] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: No bootstrap required, opened a new log
I20260812 02:43:38.553112 27245 ts_tablet_manager.cc:1397] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 02:43:38.553437 27244 tablet_bootstrap.cc:492] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: No bootstrap required, opened a new log
I20260812 02:43:38.553586 27246 tablet_bootstrap.cc:492] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: No bootstrap required, opened a new log
I20260812 02:43:38.553803 27244 ts_tablet_manager.cc:1397] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260812 02:43:38.553884 27246 ts_tablet_manager.cc:1397] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 02:43:38.561757 27244 raft_consensus.cc:348] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.562250 27244 raft_consensus.cc:374] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.562359 27244 raft_consensus.cc:729] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135224b9aab54bb18cc9319160516df3, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.563093 27244 consensus_queue.cc:260] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.564528 27246 raft_consensus.cc:348] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.564522 27245 raft_consensus.cc:348] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.564990 27246 raft_consensus.cc:374] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.564990 27245 raft_consensus.cc:374] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.565050 27244 ts_tablet_manager.cc:1428] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260812 02:43:38.565090 27246 raft_consensus.cc:729] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7b87ffb31343aaa8a2802f1b869d68, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.565090 27245 raft_consensus.cc:729] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e08c9dc21e4f0c8c4b1fcb439157c1, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.565730 27245 consensus_queue.cc:260] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.566416 27246 consensus_queue.cc:260] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.569396 27057 heartbeater.cc:503] Master 127.17.196.254:37945 was elected leader, sending a full tablet report...
I20260812 02:43:38.569628 27220 heartbeater.cc:503] Master 127.17.196.254:37945 was elected leader, sending a full tablet report...
I20260812 02:43:38.569689 27246 ts_tablet_manager.cc:1428] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260812 02:43:38.570075 27245 ts_tablet_manager.cc:1428] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent starting tablet: real 0.017s user 0.019s sys 0.001s
I20260812 02:43:38.576290 27251 raft_consensus.cc:482] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:38.576637 27251 raft_consensus.cc:504] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.578389 27251 leader_election.cc:288] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567), 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641)
I20260812 02:43:38.585224 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.578656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:46567 (local address 127.17.196.195:50869)
0812 02:43:38.578856 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:38.578875 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:38.579041 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:38.580051 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:38.580056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:38.580081 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:38.580342 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.580351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.581697 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.581706 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.583085 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.583098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.584359 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.584367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.584597 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.584638 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 02:43:38.584761 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 02:43:38.584975 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":26}
I20260812 02:43:38.585279 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.579266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:50869 (local address 127.17.196.194:46567)
0812 02:43:38.579703 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.579708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.579724 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:43:38.579781 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.579786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.579906 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.580000 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:38.580495 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.581555 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.583284 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.584177 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.584309 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.584366 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.584828 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.585021 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315}
I20260812 02:43:38.585835 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.579478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:44423 (local address 127.17.196.193:39641)
0812 02:43:38.579746 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.579752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.579774 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:43:38.579879 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.579884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.579949 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.580062 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:38.580801 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.581662 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.583581 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.584648 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.584827 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.584907 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.585344 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.585568 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:43:38.585932 27253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.578828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39641 (local address 127.17.196.195:44423)
0812 02:43:38.579621 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:38.579640 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:38.579763 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:38.580148 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:38.580154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:38.580183 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:38.580626 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.580640 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.581825 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.581829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.583316 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.583332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.584942 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.584950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.585125 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.585162 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:43:38.585291 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 02:43:38.585440 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":8,"thread_start_us":435,"threads_started":1}
I20260812 02:43:38.586802 26994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967bb861839a4d2aabc2d4768ff1ca3f" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" is_pre_election: true
I20260812 02:43:38.587087 26834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967bb861839a4d2aabc2d4768ff1ca3f" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135224b9aab54bb18cc9319160516df3" is_pre_election: true
I20260812 02:43:38.587473 26994 raft_consensus.cc:2454] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 0.
I20260812 02:43:38.587883 26834 raft_consensus.cc:2454] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 0.
I20260812 02:43:38.588270 27079 leader_election.cc:302] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7b87ffb31343aaa8a2802f1b869d68, 92e08c9dc21e4f0c8c4b1fcb439157c1; no voters:
I20260812 02:43:38.589033 27251 raft_consensus.cc:2781] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:38.589243 27251 raft_consensus.cc:482] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:38.589375 27251 raft_consensus.cc:3037] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:38.591984 27251 raft_consensus.cc:504] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.592676 27251 leader_election.cc:288] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 election: Requested vote from peers 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567), 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641)
I20260812 02:43:38.593266 26994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967bb861839a4d2aabc2d4768ff1ca3f" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7b87ffb31343aaa8a2802f1b869d68"
I20260812 02:43:38.593295 26834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967bb861839a4d2aabc2d4768ff1ca3f" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135224b9aab54bb18cc9319160516df3"
I20260812 02:43:38.593557 26994 raft_consensus.cc:3037] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:38.593561 26834 raft_consensus.cc:3037] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:38.596107 26994 raft_consensus.cc:2454] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 1.
I20260812 02:43:38.596091 26834 raft_consensus.cc:2454] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 1.
I20260812 02:43:38.596685 27079 leader_election.cc:302] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7b87ffb31343aaa8a2802f1b869d68, 92e08c9dc21e4f0c8c4b1fcb439157c1; no voters:
I20260812 02:43:38.597020 27251 raft_consensus.cc:2781] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:38.597658 27251 raft_consensus.cc:686] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 LEADER]: Becoming Leader. State: Replica: 92e08c9dc21e4f0c8c4b1fcb439157c1, State: Running, Role: LEADER
I20260812 02:43:38.598215 27251 consensus_queue.cc:237] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } }
I20260812 02:43:38.603364 26660 catalog_manager.cc:5495] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } health_report { overall_health: UNKNOWN } } }
I20260812 02:43:38.621968 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.616249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53448 (local address 127.17.196.195:34113)
0812 02:43:38.616483 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.616491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.616514 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:43:38.616814 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.616826 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.616920 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.617096 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:38.617473 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.618265 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.619618 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.620054 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.620209 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.620708 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:38.621294 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:38.621468 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.621578 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.621671 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:43:38.632103 26994 raft_consensus.cc:1261] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Refusing update from remote peer 92e08c9dc21e4f0c8c4b1fcb439157c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:38.632103 26834 raft_consensus.cc:1261] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Refusing update from remote peer 92e08c9dc21e4f0c8c4b1fcb439157c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 02:43:38.632709 27221 tablet.cc:2369] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:38.633371 27255 consensus_queue.cc:1035] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:38.633905 27251 consensus_queue.cc:1035] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:38.664625 26661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:38.664991 26661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:38.669339 26661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: Sending DeleteTablet for 3 replicas of tablet 967bb861839a4d2aabc2d4768ff1ca3f
I20260812 02:43:38.671060 27135 tablet_service.cc:1514] Processing DeleteTablet for tablet 967bb861839a4d2aabc2d4768ff1ca3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:38 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:53436
I20260812 02:43:38.671241 26973 tablet_service.cc:1514] Processing DeleteTablet for tablet 967bb861839a4d2aabc2d4768ff1ca3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:38 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:59658
I20260812 02:43:38.672070 26812 tablet_service.cc:1514] Processing DeleteTablet for tablet 967bb861839a4d2aabc2d4768ff1ca3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:38 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:55158
I20260812 02:43:38.672593 26661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:43:38.673624 26661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:38.675686 27272 tablet_replica.cc:330] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: stopping tablet replica
I20260812 02:43:38.676226 27272 raft_consensus.cc:2229] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:38.676847 27272 raft_consensus.cc:2258] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:38.679402 27272 ts_tablet_manager.cc:1905] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:38.680893 27269 tablet_replica.cc:330] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: stopping tablet replica
I20260812 02:43:38.681239 27271 tablet_replica.cc:330] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: stopping tablet replica
I20260812 02:43:38.681854 27271 raft_consensus.cc:2229] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:38.682536 27271 raft_consensus.cc:2258] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:38.685094 27271 ts_tablet_manager.cc:1905] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:38.685472 26812 tablet_service.cc:1467] Processing CreateTablet for tablet b627f850e5ca4eb295e747ed16a03fb9 (DEFAULT_TABLE table=test-table [id=b0a5357b05864eff9198e459f92c8f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:38.686039 26812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b627f850e5ca4eb295e747ed16a03fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.685101 26972 tablet_service.cc:1467] Processing CreateTablet for tablet b627f850e5ca4eb295e747ed16a03fb9 (DEFAULT_TABLE table=test-table [id=b0a5357b05864eff9198e459f92c8f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:38.689651 27272 ts_tablet_manager.cc:1918] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:38.689852 27272 log.cc:1196] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/967bb861839a4d2aabc2d4768ff1ca3f
I20260812 02:43:38.690088 26972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b627f850e5ca4eb295e747ed16a03fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.690517 27272 ts_tablet_manager.cc:1939] T 967bb861839a4d2aabc2d4768ff1ca3f P 135224b9aab54bb18cc9319160516df3: Deleting consensus metadata
I20260812 02:43:38.692759 27269 raft_consensus.cc:2229] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:43:38.692955 27271 ts_tablet_manager.cc:1918] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:38.693114 27271 log.cc:1196] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/967bb861839a4d2aabc2d4768ff1ca3f
I20260812 02:43:38.693619 27269 raft_consensus.cc:2258] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:38.693732 27271 ts_tablet_manager.cc:1939] T 967bb861839a4d2aabc2d4768ff1ca3f P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting consensus metadata
I20260812 02:43:38.694424 26648 catalog_manager.cc:4841] TS 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641): tablet 967bb861839a4d2aabc2d4768ff1ca3f (table test-table [id=1760f5bb1d1b47d38575fd3bcac536cf]) successfully deleted
I20260812 02:43:38.695660 27135 tablet_service.cc:1467] Processing CreateTablet for tablet b627f850e5ca4eb295e747ed16a03fb9 (DEFAULT_TABLE table=test-table [id=b0a5357b05864eff9198e459f92c8f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:38.696087 26646 catalog_manager.cc:4841] TS 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567): tablet 967bb861839a4d2aabc2d4768ff1ca3f (table test-table [id=1760f5bb1d1b47d38575fd3bcac536cf]) successfully deleted
I20260812 02:43:38.696297 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b627f850e5ca4eb295e747ed16a03fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.702104 27244 tablet_bootstrap.cc:492] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: Bootstrap starting.
I20260812 02:43:38.702997 27246 tablet_bootstrap.cc:492] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: Bootstrap starting.
I20260812 02:43:38.704115 27244 tablet_bootstrap.cc:654] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:38.704707 27246 tablet_bootstrap.cc:654] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:38.711469 27269 ts_tablet_manager.cc:1905] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:38.716218 27245 tablet_bootstrap.cc:492] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Bootstrap starting.
I20260812 02:43:38.717905 27246 tablet_bootstrap.cc:492] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: No bootstrap required, opened a new log
I20260812 02:43:38.718204 27246 ts_tablet_manager.cc:1397] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260812 02:43:38.718297 27245 tablet_bootstrap.cc:654] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:38.719313 27246 raft_consensus.cc:348] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.719820 27246 raft_consensus.cc:374] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.720001 27246 raft_consensus.cc:729] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7b87ffb31343aaa8a2802f1b869d68, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.720536 27246 consensus_queue.cc:260] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.721688 27246 ts_tablet_manager.cc:1428] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 02:43:38.725104 27244 tablet_bootstrap.cc:492] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: No bootstrap required, opened a new log
I20260812 02:43:38.725373 27244 ts_tablet_manager.cc:1397] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: Time spent bootstrapping tablet: real 0.023s user 0.015s sys 0.005s
I20260812 02:43:38.726408 27244 raft_consensus.cc:348] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.726785 27244 raft_consensus.cc:374] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.726943 27244 raft_consensus.cc:729] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135224b9aab54bb18cc9319160516df3, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.727367 27244 consensus_queue.cc:260] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.728003 27269 ts_tablet_manager.cc:1918] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:38.728246 27269 log.cc:1196] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/967bb861839a4d2aabc2d4768ff1ca3f
I20260812 02:43:38.729024 27269 ts_tablet_manager.cc:1939] T 967bb861839a4d2aabc2d4768ff1ca3f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting consensus metadata
I20260812 02:43:38.729059 27244 ts_tablet_manager.cc:1428] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 02:43:38.730645 27245 tablet_bootstrap.cc:492] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: No bootstrap required, opened a new log
I20260812 02:43:38.730839 27245 ts_tablet_manager.cc:1397] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.003s
I20260812 02:43:38.731364 26648 catalog_manager.cc:4841] TS 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113): tablet 967bb861839a4d2aabc2d4768ff1ca3f (table test-table [id=1760f5bb1d1b47d38575fd3bcac536cf]) successfully deleted
I20260812 02:43:38.732003 27245 raft_consensus.cc:348] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.732825 27245 raft_consensus.cc:374] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.732993 27245 raft_consensus.cc:729] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e08c9dc21e4f0c8c4b1fcb439157c1, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.733388 27245 consensus_queue.cc:260] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.734190 27245 ts_tablet_manager.cc:1428] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260812 02:43:38.769302 27059 tablet.cc:2369] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:38.800082 27264 raft_consensus.cc:482] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:38.800390 27264 raft_consensus.cc:504] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.801391 27264 leader_election.cc:288] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641), 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567)
I20260812 02:43:38.802114 26834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b627f850e5ca4eb295e747ed16a03fb9" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135224b9aab54bb18cc9319160516df3" is_pre_election: true
I20260812 02:43:38.802177 26994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b627f850e5ca4eb295e747ed16a03fb9" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" is_pre_election: true
I20260812 02:43:38.802453 26994 raft_consensus.cc:2454] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 0.
I20260812 02:43:38.802454 26834 raft_consensus.cc:2454] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 0.
I20260812 02:43:38.802999 27079 leader_election.cc:302] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7b87ffb31343aaa8a2802f1b869d68, 92e08c9dc21e4f0c8c4b1fcb439157c1; no voters:
I20260812 02:43:38.803349 27264 raft_consensus.cc:2781] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:38.803494 27264 raft_consensus.cc:482] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:38.803592 27264 raft_consensus.cc:3037] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:38.807071 27264 raft_consensus.cc:504] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.807900 27264 leader_election.cc:288] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 election: Requested vote from peers 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641), 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567)
I20260812 02:43:38.808668 26834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b627f850e5ca4eb295e747ed16a03fb9" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135224b9aab54bb18cc9319160516df3"
I20260812 02:43:38.808656 26994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b627f850e5ca4eb295e747ed16a03fb9" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7b87ffb31343aaa8a2802f1b869d68"
I20260812 02:43:38.808904 26994 raft_consensus.cc:3037] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:38.808959 26834 raft_consensus.cc:3037] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:38.812085 26994 raft_consensus.cc:2454] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 1.
I20260812 02:43:38.812143 26834 raft_consensus.cc:2454] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 1.
I20260812 02:43:38.812809 27079 leader_election.cc:302] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7b87ffb31343aaa8a2802f1b869d68, 92e08c9dc21e4f0c8c4b1fcb439157c1; no voters:
I20260812 02:43:38.813223 27264 raft_consensus.cc:2781] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:38.814735 27264 raft_consensus.cc:686] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 LEADER]: Becoming Leader. State: Replica: 92e08c9dc21e4f0c8c4b1fcb439157c1, State: Running, Role: LEADER
I20260812 02:43:38.815117 27264 consensus_queue.cc:237] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.818403 26660 catalog_manager.cc:5495] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } health_report { overall_health: HEALTHY } } }
Aug 12 02:43:38 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502616, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.254@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:38.877331 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.860702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37945 (local address 127.0.0.1:46534)
0812 02:43:38.861525 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:38.861563 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:38.861732 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:38.862113 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:38.862118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:38.862985 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:38.863259 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.863268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.864306 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.864315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.865568 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:38.865583 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.866319 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.866324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:38.866511 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.867233 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:38.867251 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:38.870082 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:38.874344 (+ 4262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:38.874362 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:38.874371 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:38.874960 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:38.875439 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:38.875446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:38.875450 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:38.875607 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:38.876233 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:38.876243 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:38.876513 (+ 270us) client_negotiation.cc:769] Sending connection context
0812 02:43:38.876715 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 02:43:38.876975 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":415,"threads_started":1}
I20260812 02:43:38.877708 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:38.860609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46534 (local address 127.17.196.254:37945)
0812 02:43:38.860859 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:38.860864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:38.861667 (+ 803us) server_negotiation.cc:407] Connection header received
0812 02:43:38.861870 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:38.861880 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:38.861936 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:38.862076 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:38.863413 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.864168 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.865719 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:38.866127 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:38.866286 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:38.870327 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:38.870333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:38.870337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:38.870372 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:38.874187 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:38.875114 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:38.875120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:38.875124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:38.875308 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:38.875735 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:38.875740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:38.875744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:38.876079 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:38.876261 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:38.876893 (+ 632us) server_negotiation.cc:299] Negotiation successful
0812 02:43:38.877182 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260812 02:43:38.884485 26660 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:46534:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 02:43:38.886114 26660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:38.894724 26812 tablet_service.cc:1467] Processing CreateTablet for tablet 668d8d3fb39d40648c0d8649d39df811 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0b89560f73d40648b2bc4e4659d13b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:43:38.895334 26812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668d8d3fb39d40648c0d8649d39df811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.895195 26972 tablet_service.cc:1467] Processing CreateTablet for tablet 668d8d3fb39d40648c0d8649d39df811 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0b89560f73d40648b2bc4e4659d13b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:43:38.895781 26972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668d8d3fb39d40648c0d8649d39df811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.901885 27244 tablet_bootstrap.cc:492] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: Bootstrap starting.
I20260812 02:43:38.902443 27135 tablet_service.cc:1467] Processing CreateTablet for tablet 668d8d3fb39d40648c0d8649d39df811 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0b89560f73d40648b2bc4e4659d13b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:43:38.903117 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668d8d3fb39d40648c0d8649d39df811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:38.903800 27244 tablet_bootstrap.cc:654] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:38.904017 27246 tablet_bootstrap.cc:492] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: Bootstrap starting.
I20260812 02:43:38.905992 27246 tablet_bootstrap.cc:654] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: Neither blocks nor log segments found. Creating new log.
W20260812 02:43:38.914460 26892 tablet.cc:2369] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:43:38.915594 27245 tablet_bootstrap.cc:492] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Bootstrap starting.
I20260812 02:43:38.916502 27244 tablet_bootstrap.cc:492] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: No bootstrap required, opened a new log
I20260812 02:43:38.916705 27244 ts_tablet_manager.cc:1397] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260812 02:43:38.917151 27246 tablet_bootstrap.cc:492] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: No bootstrap required, opened a new log
I20260812 02:43:38.917403 27246 ts_tablet_manager.cc:1397] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260812 02:43:38.917536 27244 raft_consensus.cc:348] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.917758 27245 tablet_bootstrap.cc:654] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:38.917845 27244 raft_consensus.cc:374] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.917936 27244 raft_consensus.cc:729] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135224b9aab54bb18cc9319160516df3, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.918285 27244 consensus_queue.cc:260] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.918792 27250 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:38.918851 27246 raft_consensus.cc:348] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.919193 27250 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:43:38.919333 27246 raft_consensus.cc:374] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.919477 27246 raft_consensus.cc:729] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7b87ffb31343aaa8a2802f1b869d68, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.920351 27244 ts_tablet_manager.cc:1428] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 02:43:38.919873 27246 consensus_queue.cc:260] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.922717 27246 ts_tablet_manager.cc:1428] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260812 02:43:38.922622 27252 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:38.923141 27252 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:43:38.930601 27245 tablet_bootstrap.cc:492] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: No bootstrap required, opened a new log
I20260812 02:43:38.930873 27245 ts_tablet_manager.cc:1397] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260812 02:43:38.932052 27245 raft_consensus.cc:348] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.932355 27245 raft_consensus.cc:374] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:38.932482 27245 raft_consensus.cc:729] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e08c9dc21e4f0c8c4b1fcb439157c1, State: Initialized, Role: FOLLOWER
I20260812 02:43:38.932801 27245 consensus_queue.cc:260] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:38.933444 27245 ts_tablet_manager.cc:1428] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 02:43:38.933332 27264 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:38.933740 27264 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:43:39.066599 27252 raft_consensus.cc:482] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:39.066876 27252 raft_consensus.cc:504] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.068909 27252 leader_election.cc:288] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641), 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113)
I20260812 02:43:39.078013 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.069280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:52393 (local address 127.17.196.193:39641)
0812 02:43:39.069617 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.069627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.070252 (+ 625us) server_negotiation.cc:407] Connection header received
0812 02:43:39.070685 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.070694 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.070807 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.070892 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:39.072156 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.073040 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.075278 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.076580 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.076742 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.076851 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.077527 (+ 676us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.077686 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 02:43:39.078063 27287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.069178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39641 (local address 127.17.196.194:52393)
0812 02:43:39.070143 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:39.070171 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:39.070380 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:39.071348 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:39.071351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:39.071374 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:39.071842 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.071857 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.073189 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.073196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.075072 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.075086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.077090 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.077097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.077298 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.077339 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 02:43:39.077450 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 02:43:39.077738 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":657,"threads_started":1}
I20260812 02:43:39.079084 26834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668d8d3fb39d40648c0d8649d39df811" candidate_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135224b9aab54bb18cc9319160516df3" is_pre_election: true
I20260812 02:43:39.079428 26834 raft_consensus.cc:2454] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e7b87ffb31343aaa8a2802f1b869d68 in term 0.
I20260812 02:43:39.080205 27288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.070314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:34113 (local address 127.17.196.194:54289)
0812 02:43:39.071493 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:39.071519 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:39.071742 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:39.072461 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:39.072466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:39.072495 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:39.072978 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.072995 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.074793 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.074798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.077136 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.077148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.079230 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.079237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.079423 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.079469 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 02:43:39.079656 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 02:43:39.079811 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"thread_start_us":788,"threads_started":1}
I20260812 02:43:39.080132 26916 leader_election.cc:302] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 135224b9aab54bb18cc9319160516df3, 1e7b87ffb31343aaa8a2802f1b869d68; no voters:
I20260812 02:43:39.080878 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.070717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:54289 (local address 127.17.196.195:34113)
0812 02:43:39.071226 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.071232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.071620 (+ 388us) server_negotiation.cc:407] Connection header received
0812 02:43:39.071970 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.071977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.072083 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.072168 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:39.073211 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.074643 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.077345 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.078961 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.079165 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.079255 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.080511 (+ 1256us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.080643 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361}
I20260812 02:43:39.080840 27252 raft_consensus.cc:2781] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:39.080991 27252 raft_consensus.cc:482] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:39.081094 27252 raft_consensus.cc:3037] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.081841 27167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668d8d3fb39d40648c0d8649d39df811" candidate_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" is_pre_election: true
I20260812 02:43:39.082314 27167 raft_consensus.cc:2454] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e7b87ffb31343aaa8a2802f1b869d68 in term 0.
I20260812 02:43:39.084868 27252 raft_consensus.cc:504] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.086181 27252 leader_election.cc:288] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [CANDIDATE]: Term 1 election: Requested vote from peers 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641), 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113)
I20260812 02:43:39.086477 26834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668d8d3fb39d40648c0d8649d39df811" candidate_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135224b9aab54bb18cc9319160516df3"
I20260812 02:43:39.086761 26834 raft_consensus.cc:3037] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.088958 26834 raft_consensus.cc:2454] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e7b87ffb31343aaa8a2802f1b869d68 in term 1.
I20260812 02:43:39.089558 26916 leader_election.cc:302] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 135224b9aab54bb18cc9319160516df3, 1e7b87ffb31343aaa8a2802f1b869d68; no voters:
I20260812 02:43:39.089944 27252 raft_consensus.cc:2781] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:39.089986 27167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668d8d3fb39d40648c0d8649d39df811" candidate_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1"
I20260812 02:43:39.090323 27167 raft_consensus.cc:3037] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.092645 27167 raft_consensus.cc:2454] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e7b87ffb31343aaa8a2802f1b869d68 in term 1.
I20260812 02:43:39.093068 27252 raft_consensus.cc:686] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 LEADER]: Becoming Leader. State: Replica: 1e7b87ffb31343aaa8a2802f1b869d68, State: Running, Role: LEADER
I20260812 02:43:39.093724 27252 consensus_queue.cc:237] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.096498 27289 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: TxnStatusTablet state changed. Reason: New leader 1e7b87ffb31343aaa8a2802f1b869d68. Latest consensus state: current_term: 1 leader_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:39.097018 27289 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:43:39.098228 27293 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 02:43:39.098515 27293 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 02:43:39.098691 26661 catalog_manager.cc:5495] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } health_report { overall_health: UNKNOWN } } }
I20260812 02:43:39.182964 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.161150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:46567 (local address 127.0.0.1:59674)
0812 02:43:39.161658 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:39.161681 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:39.161873 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:39.174053 (+ 12180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:39.174060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:39.174078 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:43:39.174351 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.174364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.176562 (+ 2198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.176573 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.178787 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.178828 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.180286 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.180295 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.180586 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.180698 (+ 112us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:43:39.182315 (+ 1617us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:43:39.182348 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:43:39.182415 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 02:43:39.182620 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267}
I20260812 02:43:39.183995 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.159451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.17.196.194:46567)
0812 02:43:39.173181 (+ 13730us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.173189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.173219 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:43:39.173346 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.173352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.173471 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.174583 (+ 1112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:39.174945 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.176330 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.179087 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.180077 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.180975 (+ 898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.181197 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:39.182161 (+ 964us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:39.183422 (+ 1261us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.183481 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.183581 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13552,"thread_start_us":14166,"threads_started":1}
I20260812 02:43:39.198164 26834 raft_consensus.cc:1261] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Refusing update from remote peer 1e7b87ffb31343aaa8a2802f1b869d68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:39.198499 27167 raft_consensus.cc:1261] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Refusing update from remote peer 1e7b87ffb31343aaa8a2802f1b869d68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:39.198982 27289 consensus_queue.cc:1035] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:39.199597 27252 consensus_queue.cc:1035] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:39.222268 27250 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: TxnStatusTablet state changed. Reason: New leader 1e7b87ffb31343aaa8a2802f1b869d68. Latest consensus state: current_term: 1 leader_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:39.222625 27250 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:43:39.227480 27299 mvcc.cc:205] Tried to move back new op lower bound from 7317514728225599488 to 7317514727813242880. Current Snapshot: MvccSnapshot[applied={T|T < 7317514728225599488}]
I20260812 02:43:39.228425 27264 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: TxnStatusTablet state changed. Reason: New leader 1e7b87ffb31343aaa8a2802f1b869d68. Latest consensus state: current_term: 1 leader_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:39.228856 27264 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:43:39.235455 27250 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:39.235747 27250 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 135224b9aab54bb18cc9319160516df3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:43:39.240121 27289 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:39.240564 27289 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:43:39.242367 27289 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:39.242771 27289 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 1e7b87ffb31343aaa8a2802f1b869d68: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:43:39.250207 27264 tablet_replica.cc:439] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } } }
I20260812 02:43:39.250576 27264 tablet_replica.cc:442] T 668d8d3fb39d40648c0d8649d39df811 P 92e08c9dc21e4f0c8c4b1fcb439157c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:43:39.269865 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.261592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:56489 (local address 127.17.196.194:46567)
0812 02:43:39.261781 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.261789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.262355 (+ 566us) server_negotiation.cc:407] Connection header received
0812 02:43:39.262656 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.262667 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.262776 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.262897 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:39.263328 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.264658 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.266829 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.267625 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.267780 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.268366 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:39.268523 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:39.268627 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.268847 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.269449 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:43:39.270010 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.261429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:46567 (local address 127.17.196.195:56489)
0812 02:43:39.262215 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:39.262233 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:39.262471 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:39.262911 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:39.262915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:39.262925 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:43:39.263172 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.263180 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.264864 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.264884 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.266635 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.266653 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.267851 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.267856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.267997 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.268137 (+ 140us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:43:39.268690 (+ 553us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:43:39.268704 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 02:43:39.268798 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 02:43:39.268991 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":421,"threads_started":1}
I20260812 02:43:39.269867 27264 consensus_queue.cc:1035] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:39.285346 26834 raft_consensus.cc:1261] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Refusing update from remote peer 92e08c9dc21e4f0c8c4b1fcb439157c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:39.286132 27264 consensus_queue.cc:1035] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:39.301049 27300 mvcc.cc:205] Tried to move back new op lower bound from 7317514728584671232 to 7317514726669852672. Current Snapshot: MvccSnapshot[applied={T|T < 7317514728584671232}]
I20260812 02:43:39.325320 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.317227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:32971 (local address 127.17.196.195:34113)
0812 02:43:39.317753 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.317760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.318339 (+ 579us) server_negotiation.cc:407] Connection header received
0812 02:43:39.318575 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.318582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.318677 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.318786 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:39.319637 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.320464 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.322358 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.323067 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.323230 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.324420 (+ 1190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:39.324587 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:39.324713 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.324879 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.324973 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367}
I20260812 02:43:39.325541 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.317292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:34113 (local address 127.17.196.194:32971)
0812 02:43:39.318197 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:39.318221 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:39.318407 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:39.318835 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:39.318840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:39.318854 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:43:39.319395 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.319409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.320638 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.320645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.322182 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.322197 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.323800 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.323810 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.324049 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.324143 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:43:39.324714 (+ 571us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:43:39.324730 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 02:43:39.324853 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 02:43:39.325021 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":490,"threads_started":1}
I20260812 02:43:39.368836 26660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:39.369102 26660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:39.373155 26660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: Sending DeleteTablet for 3 replicas of tablet b627f850e5ca4eb295e747ed16a03fb9
I20260812 02:43:39.374143 26812 tablet_service.cc:1514] Processing DeleteTablet for tablet b627f850e5ca4eb295e747ed16a03fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:39 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:55158
I20260812 02:43:39.374971 27320 tablet_replica.cc:330] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: stopping tablet replica
I20260812 02:43:39.375134 27135 tablet_service.cc:1514] Processing DeleteTablet for tablet b627f850e5ca4eb295e747ed16a03fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:39 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:53436
I20260812 02:43:39.375186 27320 raft_consensus.cc:2229] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:39.375511 27320 raft_consensus.cc:2258] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:39.376204 27321 tablet_replica.cc:330] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: stopping tablet replica
I20260812 02:43:39.376293 26972 tablet_service.cc:1514] Processing DeleteTablet for tablet b627f850e5ca4eb295e747ed16a03fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:39 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:59658
I20260812 02:43:39.376434 27321 raft_consensus.cc:2229] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:43:39.376974 27321 raft_consensus.cc:2258] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:39.380618 27321 ts_tablet_manager.cc:1905] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:39.385524 27320 ts_tablet_manager.cc:1905] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:39.386165 27322 tablet_replica.cc:330] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: stopping tablet replica
I20260812 02:43:39.386426 27322 raft_consensus.cc:2229] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:39.386751 27322 raft_consensus.cc:2258] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:39.388121 27322 ts_tablet_manager.cc:1905] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:39.397135 27322 ts_tablet_manager.cc:1918] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:43:39.397324 27322 log.cc:1196] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/b627f850e5ca4eb295e747ed16a03fb9
I20260812 02:43:39.397899 27322 ts_tablet_manager.cc:1939] T b627f850e5ca4eb295e747ed16a03fb9 P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting consensus metadata
I20260812 02:43:39.399384 26646 catalog_manager.cc:4841] TS 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567): tablet b627f850e5ca4eb295e747ed16a03fb9 (table test-table [id=b0a5357b05864eff9198e459f92c8f8d]) successfully deleted
I20260812 02:43:39.402573 27321 ts_tablet_manager.cc:1918] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:43:39.402740 27321 log.cc:1196] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/b627f850e5ca4eb295e747ed16a03fb9
I20260812 02:43:39.403251 27321 ts_tablet_manager.cc:1939] T b627f850e5ca4eb295e747ed16a03fb9 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting consensus metadata
I20260812 02:43:39.403975 27320 ts_tablet_manager.cc:1918] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:43:39.404186 27320 log.cc:1196] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/b627f850e5ca4eb295e747ed16a03fb9
I20260812 02:43:39.405244 26648 catalog_manager.cc:4841] TS 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113): tablet b627f850e5ca4eb295e747ed16a03fb9 (table test-table [id=b0a5357b05864eff9198e459f92c8f8d]) successfully deleted
I20260812 02:43:39.405974 27320 ts_tablet_manager.cc:1939] T b627f850e5ca4eb295e747ed16a03fb9 P 135224b9aab54bb18cc9319160516df3: Deleting consensus metadata
I20260812 02:43:39.407162 26648 catalog_manager.cc:4841] TS 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641): tablet b627f850e5ca4eb295e747ed16a03fb9 (table test-table [id=b0a5357b05864eff9198e459f92c8f8d]) successfully deleted
Aug 12 02:43:39 dist-test-slave-ss4n krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:43:39 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:39 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:43:39.480882 27330 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:39.465780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.17.196.254:37945)
0812 02:43:39.466678 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.466683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.466706 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:43:39.466773 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.466776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.466843 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.466946 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:39.469111 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.470313 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.472768 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.473279 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.473416 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.480493 (+ 7077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46546: BlockingRecv error: recv got EOF from 127.0.0.1:46546 (error 108)
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":673,"threads_started":1}
Aug 12 02:43:39 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM
I20260812 02:43:39.501147 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.484224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.17.196.254:37945)
0812 02:43:39.484576 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.484583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.484846 (+ 263us) server_negotiation.cc:407] Connection header received
0812 02:43:39.485081 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.485087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.485183 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.485305 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:39.486666 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.487432 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.488771 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.489225 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.489385 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.494119 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:39.494125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:39.494128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:39.494169 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:39.497359 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:39.498575 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:39.498580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:39.498583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:39.498786 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:39.499308 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:39.499312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:39.499314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:39.499611 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:39.499873 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.500431 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.500742 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 02:43:39.505631 26660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:43:39.506388 26660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:39.515347 26812 tablet_service.cc:1467] Processing CreateTablet for tablet 3230b2ffbf9947e7bb4168b315ac296f (DEFAULT_TABLE table=test-table [id=5fcd0569680f4862ab87456429a6ed1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:39.515597 26972 tablet_service.cc:1467] Processing CreateTablet for tablet 3230b2ffbf9947e7bb4168b315ac296f (DEFAULT_TABLE table=test-table [id=5fcd0569680f4862ab87456429a6ed1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:39.515933 26812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3230b2ffbf9947e7bb4168b315ac296f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:39.516083 26972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3230b2ffbf9947e7bb4168b315ac296f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:39.521972 27135 tablet_service.cc:1467] Processing CreateTablet for tablet 3230b2ffbf9947e7bb4168b315ac296f (DEFAULT_TABLE table=test-table [id=5fcd0569680f4862ab87456429a6ed1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:39.522482 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3230b2ffbf9947e7bb4168b315ac296f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:39.530575 27337 tablet_bootstrap.cc:492] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: Bootstrap starting.
I20260812 02:43:39.533121 27337 tablet_bootstrap.cc:654] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:39.535813 27336 tablet_bootstrap.cc:492] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: Bootstrap starting.
I20260812 02:43:39.536391 27338 tablet_bootstrap.cc:492] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Bootstrap starting.
I20260812 02:43:39.538264 27336 tablet_bootstrap.cc:654] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:39.538506 27338 tablet_bootstrap.cc:654] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:39.546345 27338 tablet_bootstrap.cc:492] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: No bootstrap required, opened a new log
I20260812 02:43:39.546685 27338 ts_tablet_manager.cc:1397] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260812 02:43:39.549898 27338 raft_consensus.cc:348] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.550536 27338 raft_consensus.cc:374] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:39.550752 27338 raft_consensus.cc:729] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e08c9dc21e4f0c8c4b1fcb439157c1, State: Initialized, Role: FOLLOWER
I20260812 02:43:39.551219 27338 consensus_queue.cc:260] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.552078 27338 ts_tablet_manager.cc:1428] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260812 02:43:39.554399 27336 tablet_bootstrap.cc:492] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: No bootstrap required, opened a new log
I20260812 02:43:39.554662 27336 ts_tablet_manager.cc:1397] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.012s
I20260812 02:43:39.555132 27337 tablet_bootstrap.cc:492] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: No bootstrap required, opened a new log
I20260812 02:43:39.555347 27337 ts_tablet_manager.cc:1397] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.016s
I20260812 02:43:39.555938 27336 raft_consensus.cc:348] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.556207 27337 raft_consensus.cc:348] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.556507 27337 raft_consensus.cc:374] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:39.556640 27337 raft_consensus.cc:729] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135224b9aab54bb18cc9319160516df3, State: Initialized, Role: FOLLOWER
I20260812 02:43:39.556950 27337 consensus_queue.cc:260] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.556340 27336 raft_consensus.cc:374] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:39.557359 27336 raft_consensus.cc:729] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7b87ffb31343aaa8a2802f1b869d68, State: Initialized, Role: FOLLOWER
I20260812 02:43:39.557708 27336 consensus_queue.cc:260] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.557905 27337 ts_tablet_manager.cc:1428] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:43:39.558395 27336 ts_tablet_manager.cc:1428] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 02:43:39.574572 27250 raft_consensus.cc:482] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:39.574863 27250 raft_consensus.cc:504] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.576399 27250 leader_election.cc:288] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567), 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113)
I20260812 02:43:39.594785 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.577428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:47763 (local address 127.17.196.194:46567)
0812 02:43:39.577653 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.577662 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.588083 (+ 10421us) server_negotiation.cc:407] Connection header received
0812 02:43:39.588356 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.588364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.588477 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.588553 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:39.589266 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.590352 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.592356 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.593757 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.593902 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.593989 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.594294 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.594464 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:43:39.594843 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.576771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:46567 (local address 127.17.196.193:47763)
0812 02:43:39.587943 (+ 11172us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:39.587962 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:39.588166 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:39.588819 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:39.588824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:39.588858 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:39.589114 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.589125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.590511 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.590518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.592128 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.592144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.593903 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.593908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.594115 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.594140 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:43:39.594261 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 02:43:39.594443 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11046,"spinlock_wait_cycles":9472,"thread_start_us":11702,"threads_started":1}
I20260812 02:43:39.595892 26994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3230b2ffbf9947e7bb4168b315ac296f" candidate_uuid: "135224b9aab54bb18cc9319160516df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" is_pre_election: true
I20260812 02:43:39.596272 26994 raft_consensus.cc:2454] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 135224b9aab54bb18cc9319160516df3 in term 0.
I20260812 02:43:39.596547 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.576973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:34113 (local address 127.17.196.193:47883)
0812 02:43:39.588802 (+ 11829us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:39.588820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:39.588954 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:39.589360 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:39.589368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:39.589405 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:43:39.589864 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.589879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.591520 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.591526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.593650 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:39.593670 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.595633 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.595641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:39.595805 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.595831 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 02:43:39.595892 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 02:43:39.596022 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11687,"spinlock_wait_cycles":896,"thread_start_us":11150,"threads_started":1}
I20260812 02:43:39.597080 26759 leader_election.cc:302] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 135224b9aab54bb18cc9319160516df3, 1e7b87ffb31343aaa8a2802f1b869d68; no voters:
I20260812 02:43:39.597289 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.577025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:47883 (local address 127.17.196.195:34113)
0812 02:43:39.577249 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.577256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.588947 (+ 11691us) server_negotiation.cc:407] Connection header received
0812 02:43:39.589095 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.589101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.589195 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.589333 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:43:39.590080 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.591328 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.593871 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.595380 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.595591 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.596699 (+ 1108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.596789 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.596950 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 02:43:39.597604 27250 raft_consensus.cc:2781] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:39.597748 27250 raft_consensus.cc:482] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:39.597862 27250 raft_consensus.cc:3037] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.597939 27166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3230b2ffbf9947e7bb4168b315ac296f" candidate_uuid: "135224b9aab54bb18cc9319160516df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" is_pre_election: true
I20260812 02:43:39.598289 27166 raft_consensus.cc:2454] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 135224b9aab54bb18cc9319160516df3 in term 0.
I20260812 02:43:39.600873 27250 raft_consensus.cc:504] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.601656 27250 leader_election.cc:288] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [CANDIDATE]: Term 1 election: Requested vote from peers 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567), 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113)
I20260812 02:43:39.602696 27166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3230b2ffbf9947e7bb4168b315ac296f" candidate_uuid: "135224b9aab54bb18cc9319160516df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1"
I20260812 02:43:39.602962 27166 raft_consensus.cc:3037] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.603260 26994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3230b2ffbf9947e7bb4168b315ac296f" candidate_uuid: "135224b9aab54bb18cc9319160516df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7b87ffb31343aaa8a2802f1b869d68"
I20260812 02:43:39.603591 26994 raft_consensus.cc:3037] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.605410 27166 raft_consensus.cc:2454] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 135224b9aab54bb18cc9319160516df3 in term 1.
I20260812 02:43:39.605958 26761 leader_election.cc:302] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 135224b9aab54bb18cc9319160516df3, 92e08c9dc21e4f0c8c4b1fcb439157c1; no voters:
I20260812 02:43:39.606318 27250 raft_consensus.cc:2781] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:39.606863 27250 raft_consensus.cc:686] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 1 LEADER]: Becoming Leader. State: Replica: 135224b9aab54bb18cc9319160516df3, State: Running, Role: LEADER
I20260812 02:43:39.607306 27250 consensus_queue.cc:237] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.608009 26994 raft_consensus.cc:2454] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 135224b9aab54bb18cc9319160516df3 in term 1.
I20260812 02:43:39.611140 26661 catalog_manager.cc:5495] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 135224b9aab54bb18cc9319160516df3 (127.17.196.193). New cstate: current_term: 1 leader_uuid: "135224b9aab54bb18cc9319160516df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } health_report { overall_health: UNKNOWN } } }
I20260812 02:43:39.635294 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.627305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.17.196.193:39641)
0812 02:43:39.628076 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.628081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.628106 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:43:39.628205 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.628209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.628365 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.628535 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:39.629091 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.630099 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.632336 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.632857 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.632990 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:39.633808 (+ 818us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:39.634480 (+ 672us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:39.634612 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:39.634787 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 02:43:39.634863 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":433,"threads_started":1}
I20260812 02:43:39.639638 26994 raft_consensus.cc:1261] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Refusing update from remote peer 135224b9aab54bb18cc9319160516df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:39.639727 27166 raft_consensus.cc:1261] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Refusing update from remote peer 135224b9aab54bb18cc9319160516df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:39.640528 27250 consensus_queue.cc:1035] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:39.640913 27347 consensus_queue.cc:1035] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:39.662776 27301 mvcc.cc:205] Tried to move back new op lower bound from 7317514730036113408 to 7317514729916280832. Current Snapshot: MvccSnapshot[applied={T|T < 7317514730036113408}]
I20260812 02:43:39.672423 26661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:39.672811 26661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:39.677142 26661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: Sending DeleteTablet for 3 replicas of tablet 3230b2ffbf9947e7bb4168b315ac296f
I20260812 02:43:39.678236 26812 tablet_service.cc:1514] Processing DeleteTablet for tablet 3230b2ffbf9947e7bb4168b315ac296f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:39 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:55158
I20260812 02:43:39.678623 26972 tablet_service.cc:1514] Processing DeleteTablet for tablet 3230b2ffbf9947e7bb4168b315ac296f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:39 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:59658
I20260812 02:43:39.678948 27320 tablet_replica.cc:330] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: stopping tablet replica
I20260812 02:43:39.679173 27320 raft_consensus.cc:2229] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:43:39.679855 27320 raft_consensus.cc:2258] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:39.680207 27322 tablet_replica.cc:330] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: stopping tablet replica
I20260812 02:43:39.680480 27322 raft_consensus.cc:2229] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:39.680763 27322 raft_consensus.cc:2258] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:39.682070 27135 tablet_service.cc:1514] Processing DeleteTablet for tablet 3230b2ffbf9947e7bb4168b315ac296f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:39 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:53436
I20260812 02:43:39.682454 27321 tablet_replica.cc:330] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: stopping tablet replica
I20260812 02:43:39.682672 27321 raft_consensus.cc:2229] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:39.682832 27322 ts_tablet_manager.cc:1905] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:39.682981 27321 raft_consensus.cc:2258] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:39.684444 27321 ts_tablet_manager.cc:1905] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:39.686317 27320 ts_tablet_manager.cc:1905] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:39.692148 27322 ts_tablet_manager.cc:1918] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:39.692373 27322 log.cc:1196] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/3230b2ffbf9947e7bb4168b315ac296f
I20260812 02:43:39.692909 27322 ts_tablet_manager.cc:1939] T 3230b2ffbf9947e7bb4168b315ac296f P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting consensus metadata
I20260812 02:43:39.694437 26646 catalog_manager.cc:4841] TS 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567): tablet 3230b2ffbf9947e7bb4168b315ac296f (table test-table [id=5fcd0569680f4862ab87456429a6ed1b]) successfully deleted
I20260812 02:43:39.699090 27321 ts_tablet_manager.cc:1918] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:39.699359 27321 log.cc:1196] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/3230b2ffbf9947e7bb4168b315ac296f
I20260812 02:43:39.699887 27321 ts_tablet_manager.cc:1939] T 3230b2ffbf9947e7bb4168b315ac296f P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting consensus metadata
I20260812 02:43:39.701370 26648 catalog_manager.cc:4841] TS 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113): tablet 3230b2ffbf9947e7bb4168b315ac296f (table test-table [id=5fcd0569680f4862ab87456429a6ed1b]) successfully deleted
I20260812 02:43:39.702503 27320 ts_tablet_manager.cc:1918] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:43:39.702772 27320 log.cc:1196] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/3230b2ffbf9947e7bb4168b315ac296f
I20260812 02:43:39.703439 27320 ts_tablet_manager.cc:1939] T 3230b2ffbf9947e7bb4168b315ac296f P 135224b9aab54bb18cc9319160516df3: Deleting consensus metadata
I20260812 02:43:39.703699 26661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:43:39.704473 26661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:43:39.705418 26648 catalog_manager.cc:4841] TS 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641): tablet 3230b2ffbf9947e7bb4168b315ac296f (table test-table [id=5fcd0569680f4862ab87456429a6ed1b]) successfully deleted
I20260812 02:43:39.713599 26812 tablet_service.cc:1467] Processing CreateTablet for tablet 5ff6595613294925b07aff6e2dfdd3f1 (DEFAULT_TABLE table=test-table [id=cade129e832445c6b4b6771a76c0fd0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:39.713994 26972 tablet_service.cc:1467] Processing CreateTablet for tablet 5ff6595613294925b07aff6e2dfdd3f1 (DEFAULT_TABLE table=test-table [id=cade129e832445c6b4b6771a76c0fd0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:39.714244 26812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ff6595613294925b07aff6e2dfdd3f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:39.714531 26972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ff6595613294925b07aff6e2dfdd3f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:39.719386 27135 tablet_service.cc:1467] Processing CreateTablet for tablet 5ff6595613294925b07aff6e2dfdd3f1 (DEFAULT_TABLE table=test-table [id=cade129e832445c6b4b6771a76c0fd0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:43:39.720290 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ff6595613294925b07aff6e2dfdd3f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:39.723959 27337 tablet_bootstrap.cc:492] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: Bootstrap starting.
I20260812 02:43:39.726011 27337 tablet_bootstrap.cc:654] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:39.729530 27338 tablet_bootstrap.cc:492] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Bootstrap starting.
I20260812 02:43:39.730329 27336 tablet_bootstrap.cc:492] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: Bootstrap starting.
I20260812 02:43:39.731360 27338 tablet_bootstrap.cc:654] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:39.732069 27336 tablet_bootstrap.cc:654] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:39.733183 27337 tablet_bootstrap.cc:492] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: No bootstrap required, opened a new log
I20260812 02:43:39.733421 27337 ts_tablet_manager.cc:1397] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260812 02:43:39.734622 27337 raft_consensus.cc:348] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.735014 27337 raft_consensus.cc:374] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:39.735121 27337 raft_consensus.cc:729] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135224b9aab54bb18cc9319160516df3, State: Initialized, Role: FOLLOWER
I20260812 02:43:39.735443 27337 consensus_queue.cc:260] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.736090 27337 ts_tablet_manager.cc:1428] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260812 02:43:39.738248 27338 tablet_bootstrap.cc:492] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: No bootstrap required, opened a new log
I20260812 02:43:39.738387 27336 tablet_bootstrap.cc:492] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: No bootstrap required, opened a new log
I20260812 02:43:39.738547 27338 ts_tablet_manager.cc:1397] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 02:43:39.738586 27336 ts_tablet_manager.cc:1397] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.001s
I20260812 02:43:39.739486 27336 raft_consensus.cc:348] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.739761 27336 raft_consensus.cc:374] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:39.739837 27336 raft_consensus.cc:729] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7b87ffb31343aaa8a2802f1b869d68, State: Initialized, Role: FOLLOWER
I20260812 02:43:39.740155 27336 consensus_queue.cc:260] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.740763 27336 ts_tablet_manager.cc:1428] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:43:39.741014 27338 raft_consensus.cc:348] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.741375 27338 raft_consensus.cc:374] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:39.741525 27338 raft_consensus.cc:729] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e08c9dc21e4f0c8c4b1fcb439157c1, State: Initialized, Role: FOLLOWER
I20260812 02:43:39.741847 27338 consensus_queue.cc:260] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.742564 27338 ts_tablet_manager.cc:1428] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260812 02:43:39.913793 27315 raft_consensus.cc:482] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:43:39.914127 27315 raft_consensus.cc:504] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.915139 27315 leader_election.cc:288] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641), 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567)
I20260812 02:43:39.916008 26994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff6595613294925b07aff6e2dfdd3f1" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" is_pre_election: true
I20260812 02:43:39.915933 26833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff6595613294925b07aff6e2dfdd3f1" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135224b9aab54bb18cc9319160516df3" is_pre_election: true
I20260812 02:43:39.916302 26833 raft_consensus.cc:2454] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 0.
I20260812 02:43:39.916301 26994 raft_consensus.cc:2454] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 0.
I20260812 02:43:39.916934 27079 leader_election.cc:302] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7b87ffb31343aaa8a2802f1b869d68, 92e08c9dc21e4f0c8c4b1fcb439157c1; no voters:
I20260812 02:43:39.917346 27315 raft_consensus.cc:2781] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:43:39.917510 27315 raft_consensus.cc:482] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:43:39.917618 27315 raft_consensus.cc:3037] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.921085 27315 raft_consensus.cc:504] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.921984 27315 leader_election.cc:288] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 election: Requested vote from peers 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641), 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567)
I20260812 02:43:39.922652 26833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff6595613294925b07aff6e2dfdd3f1" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135224b9aab54bb18cc9319160516df3"
I20260812 02:43:39.923064 26833 raft_consensus.cc:3037] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.923158 26994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ff6595613294925b07aff6e2dfdd3f1" candidate_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7b87ffb31343aaa8a2802f1b869d68"
I20260812 02:43:39.923470 26994 raft_consensus.cc:3037] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:39.926730 26994 raft_consensus.cc:2454] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 1.
I20260812 02:43:39.927248 26833 raft_consensus.cc:2454] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e08c9dc21e4f0c8c4b1fcb439157c1 in term 1.
I20260812 02:43:39.927394 27079 leader_election.cc:302] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7b87ffb31343aaa8a2802f1b869d68, 92e08c9dc21e4f0c8c4b1fcb439157c1; no voters:
I20260812 02:43:39.927955 27315 raft_consensus.cc:2781] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:39.928524 27315 raft_consensus.cc:686] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 LEADER]: Becoming Leader. State: Replica: 92e08c9dc21e4f0c8c4b1fcb439157c1, State: Running, Role: LEADER
I20260812 02:43:39.928912 27315 consensus_queue.cc:237] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } }
I20260812 02:43:39.932313 26661 catalog_manager.cc:5495] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e08c9dc21e4f0c8c4b1fcb439157c1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 34113 } health_report { overall_health: HEALTHY } } }
I20260812 02:43:40.001113 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:39.995170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.17.196.195:34113)
0812 02:43:39.995390 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:39.995398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:39.995422 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:43:39.995606 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:39.995612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:39.995708 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:39.995815 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:43:39.996209 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.997471 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.998902 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:39.999661 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:39.999816 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:40.000260 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:43:40.000421 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:43:40.000545 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:40.000710 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 02:43:40.000809 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:43:40.012882 26994 raft_consensus.cc:1261] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Refusing update from remote peer 92e08c9dc21e4f0c8c4b1fcb439157c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:40.013597 26833 raft_consensus.cc:1261] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Refusing update from remote peer 92e08c9dc21e4f0c8c4b1fcb439157c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:43:40.013659 27357 consensus_queue.cc:1035] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7b87ffb31343aaa8a2802f1b869d68" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:40.014297 27357 consensus_queue.cc:1035] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "135224b9aab54bb18cc9319160516df3" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:43:40.085254 26660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:40.085644 26660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:43:40.089570 26660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1670d18175594aa2bf3a651933ef3ec0: Sending DeleteTablet for 3 replicas of tablet 5ff6595613294925b07aff6e2dfdd3f1
I20260812 02:43:40.090723 26972 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ff6595613294925b07aff6e2dfdd3f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:40 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:59658
I20260812 02:43:40.090716 26812 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ff6595613294925b07aff6e2dfdd3f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:40 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:55158
I20260812 02:43:40.091061 27322 tablet_replica.cc:330] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: stopping tablet replica
I20260812 02:43:40.091075 27320 tablet_replica.cc:330] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: stopping tablet replica
I20260812 02:43:40.091255 27322 raft_consensus.cc:2229] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:40.091265 27320 raft_consensus.cc:2229] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:43:40.091482 27322 raft_consensus.cc:2258] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:40.091497 27320 raft_consensus.cc:2258] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:40.092419 27135 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ff6595613294925b07aff6e2dfdd3f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:43:40 UTC) from {username='oryx', principal='oryx/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:53436
I20260812 02:43:40.092794 27321 tablet_replica.cc:330] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: stopping tablet replica
I20260812 02:43:40.092988 27320 ts_tablet_manager.cc:1905] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:40.092980 27322 ts_tablet_manager.cc:1905] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:40.092998 27321 raft_consensus.cc:2229] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:43:40.093402 27321 raft_consensus.cc:2258] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:43:40.094838 27321 ts_tablet_manager.cc:1905] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:43:40.100615 27322 ts_tablet_manager.cc:1918] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:43:40.100811 27322 log.cc:1196] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/5ff6595613294925b07aff6e2dfdd3f1
I20260812 02:43:40.100931 27320 ts_tablet_manager.cc:1918] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:43:40.101102 27320 log.cc:1196] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/5ff6595613294925b07aff6e2dfdd3f1
I20260812 02:43:40.101296 27322 ts_tablet_manager.cc:1939] T 5ff6595613294925b07aff6e2dfdd3f1 P 1e7b87ffb31343aaa8a2802f1b869d68: Deleting consensus metadata
I20260812 02:43:40.101670 27320 ts_tablet_manager.cc:1939] T 5ff6595613294925b07aff6e2dfdd3f1 P 135224b9aab54bb18cc9319160516df3: Deleting consensus metadata
I20260812 02:43:40.102859 26648 catalog_manager.cc:4841] TS 135224b9aab54bb18cc9319160516df3 (127.17.196.193:39641): tablet 5ff6595613294925b07aff6e2dfdd3f1 (table test-table [id=cade129e832445c6b4b6771a76c0fd0e]) successfully deleted
I20260812 02:43:40.102795 26646 catalog_manager.cc:4841] TS 1e7b87ffb31343aaa8a2802f1b869d68 (127.17.196.194:46567): tablet 5ff6595613294925b07aff6e2dfdd3f1 (table test-table [id=cade129e832445c6b4b6771a76c0fd0e]) successfully deleted
I20260812 02:43:40.103911 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 26740
I20260812 02:43:40.105963 27321 ts_tablet_manager.cc:1918] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:43:40.106319 27321 log.cc:1196] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/5ff6595613294925b07aff6e2dfdd3f1
I20260812 02:43:40.106884 27321 ts_tablet_manager.cc:1939] T 5ff6595613294925b07aff6e2dfdd3f1 P 92e08c9dc21e4f0c8c4b1fcb439157c1: Deleting consensus metadata
I20260812 02:43:40.107971 26648 catalog_manager.cc:4841] TS 92e08c9dc21e4f0c8c4b1fcb439157c1 (127.17.196.195:34113): tablet 5ff6595613294925b07aff6e2dfdd3f1 (table test-table [id=cade129e832445c6b4b6771a76c0fd0e]) successfully deleted
Aug 12 02:43:40 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.193@KRBTEST.COM
I20260812 02:43:40.124446 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:40.105004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55182 (local address 127.17.196.193:39641)
0812 02:43:40.105223 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:40.105230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:40.106150 (+ 920us) server_negotiation.cc:407] Connection header received
0812 02:43:40.106346 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:40.106352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:40.106441 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:40.106595 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:40.108472 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:40.109261 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:40.111515 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:40.111960 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:40.112067 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:40.117680 (+ 5613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:40.117700 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:40.117718 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:40.117768 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:40.121081 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:40.122040 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:40.122049 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:40.122051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:40.122228 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:40.122682 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:40.122692 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:40.122695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:40.123091 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:40.123482 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:40.123825 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 02:43:40.124073 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:43:40.125190 26886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:40.268510 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 26740
I20260812 02:43:40.297346 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 26895
Aug 12 02:43:40 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.194@KRBTEST.COM
I20260812 02:43:40.315656 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:40.298272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.17.196.194:46567)
0812 02:43:40.298895 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:40.298902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:40.298923 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:43:40.298989 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:40.298993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:40.299081 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:40.299209 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:40.300550 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:40.301357 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:40.302919 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:40.303367 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:40.303554 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:40.308709 (+ 5155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:40.308729 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:40.308746 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:40.308794 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:40.312449 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:40.313284 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:40.313293 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:40.313295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:40.313590 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:40.314056 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:40.314061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:40.314063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:40.314552 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:40.314820 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:40.315140 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 02:43:40.315342 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":387,"threads_started":1}
I20260812 02:43:40.316449 27051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:40.446343 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 26895
I20260812 02:43:40.472007 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 27062
Aug 12 02:43:40 dist-test-slave-ss4n krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.196.195@KRBTEST.COM
I20260812 02:43:40.490314 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:40.472953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53462 (local address 127.17.196.195:34113)
0812 02:43:40.473164 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:40.473173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:40.473194 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:43:40.473260 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:40.473264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:40.473346 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:40.473460 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:40.474969 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:40.475967 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:40.477405 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:40.478033 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:40.478145 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:40.483457 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:40.483476 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:40.483494 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:40.483541 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:40.487071 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:40.487873 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:40.487881 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:40.487884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:40.488048 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:40.488448 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:40.488452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:40.488455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:40.488896 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:40.489202 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:40.489754 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 02:43:40.489990 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:43:40.491178 27214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:40.628676 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 27062
I20260812 02:43:40.656210 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 26628
I20260812 02:43:40.657437 26706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:40.780200 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 26628
2026-08-12T02:43:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6400 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:43:40 dist-test-slave-ss4n krb5kdc[27375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:43:40 dist-test-slave-ss4n krb5kdc[27375](info): set up 2 sockets
Aug 12 02:43:40 dist-test-slave-ss4n krb5kdc[27375](info): commencing operation
krb5kdc: starting...
W20260812 02:43:42.059566 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:43:42 dist-test-slave-ss4n krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:43:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:43:42.362196 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42443
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:42443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:42.719512 27390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:42.719853 27390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:42.720021 27390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:42.727875 27390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:42.728010 27390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:42.728120 27390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:42.728229 27390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:42.728320 27390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:42.739027 27390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:42443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:42.740453 27390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:42.741947 27390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:42.755664 27398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:42.755640 27396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:42.756191 27395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:42.756780 27390 server_base.cc:1034] running on GCE node
I20260812 02:43:42.757731 27390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:42.759406 27390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:42.760702 27390 hybrid_clock.cc:648] HybridClock initialized: now 1786502622760625 us; error 71 us; skew 500 ppm
I20260812 02:43:42.761155 27390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T02:43:42Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2052 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:43:42 dist-test-slave-ss4n krb5kdc[27404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:43:42 dist-test-slave-ss4n krb5kdc[27404](info): set up 2 sockets
Aug 12 02:43:42 dist-test-slave-ss4n krb5kdc[27404](info): commencing operation
krb5kdc: starting...
W20260812 02:43:44.122260 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:43:44 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:43:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:43:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:43:44.430430 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46765
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:44.806623 27419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:44.806970 27419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:44.807150 27419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:44.814733 27419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:44.814862 27419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:44.814957 27419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:44.815080 27419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:44.815147 27419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:44.825333 27419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:46765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:44.826910 27419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:44.828330 27419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:44.841517 27424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:44.841607 27428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:44.841610 27425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:44.842577 27419 server_base.cc:1034] running on GCE node
I20260812 02:43:44.843425 27419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:44.845002 27419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:44.846267 27419 hybrid_clock.cc:648] HybridClock initialized: now 1786502624846223 us; error 62 us; skew 500 ppm
I20260812 02:43:44.846782 27419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:44 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502624, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:44.850695 27419 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:43:44.852631 27419 webserver.cc:460] Webserver started at http://127.17.196.254:39181/ using document root <none> and password file <none>
I20260812 02:43:44.853413 27419 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:44.853632 27419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:44.853998 27419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:44.857398 27419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "81778cce2a924d17bc4466866555a89d"
format_stamp: "Formatted at 2026-08-12 02:43:44 on dist-test-slave-ss4n"
server_key: "8a41ab0020056bdffb9a7f623e8baef2"
server_key_iv: "da7e65bdfe0c7ef49c4b61896135015a"
server_key_version: "encryptionkey@0"
I20260812 02:43:44.858394 27419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "81778cce2a924d17bc4466866555a89d"
format_stamp: "Formatted at 2026-08-12 02:43:44 on dist-test-slave-ss4n"
server_key: "8a41ab0020056bdffb9a7f623e8baef2"
server_key_iv: "da7e65bdfe0c7ef49c4b61896135015a"
server_key_version: "encryptionkey@0"
I20260812 02:43:44.864673 27419 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:43:44.869084 27436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:44.871095 27419 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260812 02:43:44.871356 27419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "81778cce2a924d17bc4466866555a89d"
format_stamp: "Formatted at 2026-08-12 02:43:44 on dist-test-slave-ss4n"
server_key: "8a41ab0020056bdffb9a7f623e8baef2"
server_key_iv: "da7e65bdfe0c7ef49c4b61896135015a"
server_key_version: "encryptionkey@0"
I20260812 02:43:44.871629 27419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:44.892614 27419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:44.897840 27419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:44.898221 27419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:44.921630 27419 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:46765
I20260812 02:43:44.921774 27496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:46765 every 8 connection(s)
I20260812 02:43:44.923341 27419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:44.927997 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:43:44.932564 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 27419
I20260812 02:43:44.932817 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:43:44.933722 18195 external_mini_cluster.cc:1428] Setting key a06b812a0a2f41f5d1b0554814a184d8
I20260812 02:43:44.941233 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Bootstrap starting.
I20260812 02:43:44.944593 27497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Neither blocks nor log segments found. Creating new log.
I20260812 02:43:44.945734 27497 log.cc:824] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:44.949744 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: No bootstrap required, opened a new log
Aug 12 02:43:44 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:44.958676 27497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } }
I20260812 02:43:44.959149 27497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:43:44.959304 27497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81778cce2a924d17bc4466866555a89d, State: Initialized, Role: FOLLOWER
I20260812 02:43:44.960083 27497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } }
I20260812 02:43:44.960392 27497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:44.960572 27497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:44.960783 27497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:43:44.961105 27502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:44.936210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.17.196.254:46765)
0812 02:43:44.937101 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:44.937115 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:44.937162 (+ 47us) server_negotiation.cc:407] Connection header received
0812 02:43:44.939853 (+ 2691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:44.939882 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:44.940347 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:44.940809 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:44.942540 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:44.943931 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:44.945309 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:44.946159 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:44.946396 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:44.951966 (+ 5570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:44.951975 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:44.951992 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:44.952048 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:44.955915 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:44.956840 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:44.956857 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:44.956869 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:44.957072 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:44.957575 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:44.957581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:44.957585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:44.958264 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:44.958616 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:44.959183 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 02:43:44.959676 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":404,"threads_started":1}
I20260812 02:43:44.964103 27497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } }
I20260812 02:43:44.964757 27497 leader_election.cc:302] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81778cce2a924d17bc4466866555a89d; no voters:
I20260812 02:43:44.965341 27497 leader_election.cc:288] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:43:44.965541 27504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:43:44.966449 27504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 1 LEADER]: Becoming Leader. State: Replica: 81778cce2a924d17bc4466866555a89d, State: Running, Role: LEADER
I20260812 02:43:44.967301 27504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } }
I20260812 02:43:44.967731 27497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:44.973029 27505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81778cce2a924d17bc4466866555a89d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } } }
I20260812 02:43:44.973059 27506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81778cce2a924d17bc4466866555a89d. Latest consensus state: current_term: 1 leader_uuid: "81778cce2a924d17bc4466866555a89d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } } }
I20260812 02:43:44.973568 27505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:44.973587 27506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:44.982510 27517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:44.988957 27517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:44.999826 27517 catalog_manager.cc:1347] Generated new cluster ID: a0b1786329bf47c4b442c21e5e045079
I20260812 02:43:44.999984 27517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:45.015286 27517 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:43:45.016860 27517 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:45.028785 27517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Generated new TSK 0
I20260812 02:43:45.029942 27517 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:43:45.118098 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46765
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:45.500226 27526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:45.500550 27526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:45.500705 27526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:45.508327 27526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:45.508455 27526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:45.508657 27526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:45.518905 27526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:45.520319 27526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:45.521746 27526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:45.535776 27533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:45.535852 27532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:45.535849 27536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:45.537058 27526 server_base.cc:1034] running on GCE node
I20260812 02:43:45.537819 27526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:45.538779 27526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:45.540016 27526 hybrid_clock.cc:648] HybridClock initialized: now 1786502625539947 us; error 55 us; skew 500 ppm
I20260812 02:43:45.540351 27526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:45 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502625, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:45.544169 27526 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:43:45.545992 27526 webserver.cc:460] Webserver started at http://127.17.196.193:38563/ using document root <none> and password file <none>
I20260812 02:43:45.546756 27526 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:45.546869 27526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:45.547166 27526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:45.550585 27526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "883261ecd6e74c24bdee9acf394d7016"
format_stamp: "Formatted at 2026-08-12 02:43:45 on dist-test-slave-ss4n"
server_key: "398e48eb6b39fb6dcade2d7dde6df759"
server_key_iv: "c030ffced8129e68caa8df2f6213d7e0"
server_key_version: "encryptionkey@0"
I20260812 02:43:45.551494 27526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "883261ecd6e74c24bdee9acf394d7016"
format_stamp: "Formatted at 2026-08-12 02:43:45 on dist-test-slave-ss4n"
server_key: "398e48eb6b39fb6dcade2d7dde6df759"
server_key_iv: "c030ffced8129e68caa8df2f6213d7e0"
server_key_version: "encryptionkey@0"
I20260812 02:43:45.557796 27526 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 02:43:45.562681 27543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:45.564380 27526 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:43:45.564661 27526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "883261ecd6e74c24bdee9acf394d7016"
format_stamp: "Formatted at 2026-08-12 02:43:45 on dist-test-slave-ss4n"
server_key: "398e48eb6b39fb6dcade2d7dde6df759"
server_key_iv: "c030ffced8129e68caa8df2f6213d7e0"
server_key_version: "encryptionkey@0"
I20260812 02:43:45.564906 27526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:45.592625 27526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:45.597710 27526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:45.598049 27526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:45.599370 27526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:45.600924 27526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:45.601033 27526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:45.601203 27526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:45.601279 27526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:45.641290 27526 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:37291
I20260812 02:43:45.641386 27673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:37291 every 8 connection(s)
I20260812 02:43:45.642766 27526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:45.651002 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 27526
I20260812 02:43:45.651238 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:43:45.652005 18195 external_mini_cluster.cc:1428] Setting key 13a462c14113d147e0f40757f447dd73
Aug 12 02:43:45 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502625, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:45.668491 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:45.645743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:57081 (local address 127.17.196.254:46765)
0812 02:43:45.646229 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:45.646234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:45.647364 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 02:43:45.649373 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:45.649379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:45.649532 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:45.649664 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:45.651342 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:45.652098 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:45.654574 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:45.655046 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:45.655172 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:45.660007 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:45.660017 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:45.660026 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:45.660089 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:45.664242 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:45.665199 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:45.665209 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:45.665216 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:45.665393 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:45.665969 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:45.665977 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:45.665984 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:45.666454 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:45.666642 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:45.667860 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0812 02:43:45.668145 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":241,"threads_started":1}
I20260812 02:43:45.670058 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:45.646366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:57081)
0812 02:43:45.647158 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:45.647217 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:45.649042 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:45.649929 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:45.649938 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:45.650493 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:45.651143 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:45.651174 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:45.652323 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:45.652330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:45.654352 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:45.654379 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:45.655329 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:45.655339 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:45.655720 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:45.656582 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:45.656624 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:45.659698 (+ 3074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:45.664444 (+ 4746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:45.664453 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:45.664527 (+ 74us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:45.665034 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:45.665601 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:45.665604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:45.665607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:45.665827 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:45.666632 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:45.666639 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:45.667240 (+ 601us) client_negotiation.cc:769] Sending connection context
0812 02:43:45.667616 (+ 376us) client_negotiation.cc:240] Negotiation successful
0812 02:43:45.668330 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":313,"threads_started":1}
I20260812 02:43:45.672024 27674 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46765
I20260812 02:43:45.672433 27674 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:45.673302 27674 heartbeater.cc:511] Master 127.17.196.254:46765 requested a full tablet report, sending...
I20260812 02:43:45.675870 27456 ts_manager.cc:194] Registered new tserver with Master: 883261ecd6e74c24bdee9acf394d7016 (127.17.196.193:37291)
I20260812 02:43:45.678205 27456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:57081
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:43:45.723786 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46765
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:46.087575 27681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:46.087888 27681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:46.088068 27681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:46.095794 27681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:46.095933 27681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:46.096131 27681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:46.106535 27681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:46.107931 27681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:46.109299 27681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:46.123452 27688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:46.123435 27689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:46.123620 27691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:46.124588 27681 server_base.cc:1034] running on GCE node
I20260812 02:43:46.125265 27681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:46.126371 27681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:46.127626 27681 hybrid_clock.cc:648] HybridClock initialized: now 1786502626127554 us; error 72 us; skew 500 ppm
I20260812 02:43:46.127995 27681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:46 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502626, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:46.131559 27681 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:43:46.133291 27681 webserver.cc:460] Webserver started at http://127.17.196.194:37571/ using document root <none> and password file <none>
I20260812 02:43:46.134100 27681 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:46.134225 27681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:46.134492 27681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:46.137825 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "42b3986b8d0642e0ac481b1b70e94c29"
format_stamp: "Formatted at 2026-08-12 02:43:46 on dist-test-slave-ss4n"
server_key: "83c60a466885df6624458f72badd1d45"
server_key_iv: "6a730ef84ea4cee9fee683908bf101a7"
server_key_version: "encryptionkey@0"
I20260812 02:43:46.138669 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "42b3986b8d0642e0ac481b1b70e94c29"
format_stamp: "Formatted at 2026-08-12 02:43:46 on dist-test-slave-ss4n"
server_key: "83c60a466885df6624458f72badd1d45"
server_key_iv: "6a730ef84ea4cee9fee683908bf101a7"
server_key_version: "encryptionkey@0"
I20260812 02:43:46.144807 27681 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:43:46.148888 27699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:46.150600 27681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 02:43:46.150804 27681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "42b3986b8d0642e0ac481b1b70e94c29"
format_stamp: "Formatted at 2026-08-12 02:43:46 on dist-test-slave-ss4n"
server_key: "83c60a466885df6624458f72badd1d45"
server_key_iv: "6a730ef84ea4cee9fee683908bf101a7"
server_key_version: "encryptionkey@0"
I20260812 02:43:46.151036 27681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:46.168938 27681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:46.174322 27681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:46.174654 27681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:46.175809 27681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:46.177286 27681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:46.177397 27681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:46.177583 27681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:46.177668 27681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:46.213691 27681 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:42181
I20260812 02:43:46.213791 27829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:42181 every 8 connection(s)
I20260812 02:43:46.215076 27681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:46.224694 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 27681
I20260812 02:43:46.224957 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:43:46.225700 18195 external_mini_cluster.cc:1428] Setting key a9ec206c42aff54c0e6fa55890f7376f
Aug 12 02:43:46 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502626, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:46.240289 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:46.218036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:35383 (local address 127.17.196.254:46765)
0812 02:43:46.218632 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:46.218637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:46.219713 (+ 1076us) server_negotiation.cc:407] Connection header received
0812 02:43:46.221894 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:46.221899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:46.221986 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:46.222094 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:46.223887 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:46.224815 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:46.227373 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:46.228013 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:46.228113 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:46.232369 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:46.232379 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:46.232386 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:46.232438 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:46.236483 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:46.237406 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:46.237415 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:46.237421 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:46.237638 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:46.238303 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:46.238310 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:46.238316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:46.238727 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:46.238885 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:46.239799 (+ 914us) server_negotiation.cc:299] Negotiation successful
0812 02:43:46.240009 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":270,"threads_started":1}
I20260812 02:43:46.242190 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:46.218634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:35383)
0812 02:43:46.219445 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:46.219568 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:46.221572 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:46.222371 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:46.222380 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:46.223037 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:46.223679 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:46.223699 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:46.225087 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:46.225094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:46.227183 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:46.227201 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:46.228300 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:46.228308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:46.228552 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:46.229409 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:46.229437 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:46.232054 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:46.236679 (+ 4625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:46.236688 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:46.236704 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:46.237242 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:46.237770 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:46.237774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:46.237777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:46.238148 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:46.238868 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:46.238875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:46.239317 (+ 442us) client_negotiation.cc:769] Sending connection context
0812 02:43:46.239696 (+ 379us) client_negotiation.cc:240] Negotiation successful
0812 02:43:46.240413 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":325,"threads_started":1}
I20260812 02:43:46.243937 27830 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46765
I20260812 02:43:46.244302 27830 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:46.245092 27830 heartbeater.cc:511] Master 127.17.196.254:46765 requested a full tablet report, sending...
I20260812 02:43:46.246908 27456 ts_manager.cc:194] Registered new tserver with Master: 42b3986b8d0642e0ac481b1b70e94c29 (127.17.196.194:42181)
I20260812 02:43:46.247908 27456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:35383
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:43:46.298127 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46765
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:46.672086 27837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:46.672451 27837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:46.672657 27837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:46.681274 27837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:46.681411 27837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:46.681670 27837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:46.681913 27674 heartbeater.cc:503] Master 127.17.196.254:46765 was elected leader, sending a full tablet report...
I20260812 02:43:46.692790 27837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:46.694294 27837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:46.695783 27837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:46.710242 27843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:46.710134 27844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:46.710153 27846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:46.711417 27837 server_base.cc:1034] running on GCE node
I20260812 02:43:46.712195 27837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:46.713279 27837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:46.714546 27837 hybrid_clock.cc:648] HybridClock initialized: now 1786502626714493 us; error 39 us; skew 500 ppm
I20260812 02:43:46.714916 27837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:46 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502626, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:46.718897 27837 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:43:46.720585 27837 webserver.cc:460] Webserver started at http://127.17.196.195:35829/ using document root <none> and password file <none>
I20260812 02:43:46.721350 27837 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:46.721561 27837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:46.721911 27837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:43:46.725638 27837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "d6a3a02d3cf340d6af57c5089f7281b0"
format_stamp: "Formatted at 2026-08-12 02:43:46 on dist-test-slave-ss4n"
server_key: "4d8cdd8610a9546893e4cd27b12a27f9"
server_key_iv: "77faf091790dd7ab06d6a4d57b7734cf"
server_key_version: "encryptionkey@0"
I20260812 02:43:46.726573 27837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "d6a3a02d3cf340d6af57c5089f7281b0"
format_stamp: "Formatted at 2026-08-12 02:43:46 on dist-test-slave-ss4n"
server_key: "4d8cdd8610a9546893e4cd27b12a27f9"
server_key_iv: "77faf091790dd7ab06d6a4d57b7734cf"
server_key_version: "encryptionkey@0"
I20260812 02:43:46.732595 27837 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260812 02:43:46.736532 27856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:46.738222 27837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:43:46.738435 27837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "d6a3a02d3cf340d6af57c5089f7281b0"
format_stamp: "Formatted at 2026-08-12 02:43:46 on dist-test-slave-ss4n"
server_key: "4d8cdd8610a9546893e4cd27b12a27f9"
server_key_iv: "77faf091790dd7ab06d6a4d57b7734cf"
server_key_version: "encryptionkey@0"
I20260812 02:43:46.738677 27837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:46.760620 27837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:46.766373 27837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:46.766762 27837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:46.768038 27837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:46.769951 27837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:46.770087 27837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:46.770269 27837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:46.770358 27837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:46.810807 27837 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:46511
I20260812 02:43:46.810921 27985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:46511 every 8 connection(s)
I20260812 02:43:46.812312 27837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:46.820472 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 27837
I20260812 02:43:46.820778 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:43:46.821261 18195 external_mini_cluster.cc:1428] Setting key 67a6f7ac3a837e42b9cee70d9b000dd3
Aug 12 02:43:46 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502626, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:43:46.837556 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:46.815426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:50407 (local address 127.17.196.254:46765)
0812 02:43:46.815987 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:46.815995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:46.817103 (+ 1108us) server_negotiation.cc:407] Connection header received
0812 02:43:46.819111 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:46.819116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:46.819229 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:46.819372 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:46.821097 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:46.822150 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:46.824458 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:46.825174 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:46.825319 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:46.829801 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:46.829807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:46.829810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:46.829879 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:46.832997 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:46.834066 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:46.834071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:46.834073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:46.834222 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:46.834793 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:46.834797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:46.834800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:46.835250 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:46.835435 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:46.836865 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0812 02:43:46.837149 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":276,"threads_started":1}
I20260812 02:43:46.838676 27989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:46.816067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:50407)
0812 02:43:46.816887 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:46.816948 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:46.818797 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:46.819633 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:46.819642 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:46.820197 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:46.820890 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:46.820933 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:46.822328 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:46.822335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:46.824278 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:46.824294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:46.825471 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:46.825499 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:46.825816 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:46.826714 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:46.826746 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:46.829357 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:43:46.833184 (+ 3827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:46.833193 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:46.833208 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:46.833881 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:46.834351 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:43:46.834354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:43:46.834357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:43:46.834633 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:43:46.835429 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:43:46.835437 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:43:46.836096 (+ 659us) client_negotiation.cc:769] Sending connection context
0812 02:43:46.836507 (+ 411us) client_negotiation.cc:240] Negotiation successful
0812 02:43:46.837084 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":348,"threads_started":1}
I20260812 02:43:46.840281 27986 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46765
I20260812 02:43:46.840637 27986 heartbeater.cc:464] Registering TS with master...
I20260812 02:43:46.841449 27986 heartbeater.cc:511] Master 127.17.196.254:46765 requested a full tablet report, sending...
I20260812 02:43:46.843289 27456 ts_manager.cc:194] Registered new tserver with Master: d6a3a02d3cf340d6af57c5089f7281b0 (127.17.196.195:46511)
I20260812 02:43:46.844303 27456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:50407
I20260812 02:43:46.850976 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab.
Entry for principal oryx/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab.
WARNING: no policy specified for oryx/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab.
Entry for principal oryx/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab.
WARNING: no policy specified for oryx/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab.
Entry for principal oryx/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab.
I20260812 02:43:47.054127 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 27526
Aug 12 02:43:47 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:43:47.076804 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:47.055748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.17.196.193:37291)
0812 02:43:47.056578 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:47.056586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:47.056617 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:43:47.056683 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:47.056691 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:47.056934 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:47.057079 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:47.058935 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:47.060888 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:47.062464 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:47.063207 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:47.063345 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:47.068681 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:47.068699 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:47.068716 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:47.068773 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:47.072935 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:47.074031 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:47.074041 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:47.074043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:47.074224 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:47.074662 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:47.074666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:47.074668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:47.075098 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:47.075672 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:47.076031 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 02:43:47.076366 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":441,"threads_started":1}
I20260812 02:43:47.078414 27668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:47.164433 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 27526
I20260812 02:43:47.186414 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 27681
Aug 12 02:43:47 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:43:47.204375 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:47.187514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60404 (local address 127.17.196.194:42181)
0812 02:43:47.188090 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:47.188098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:47.188124 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:43:47.188188 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:47.188195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:47.188415 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:47.188608 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:47.190056 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:47.191143 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:47.192440 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:47.193024 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:47.193187 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:47.197760 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:47.197774 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:47.197792 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:47.197832 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:47.200917 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:47.201974 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:47.201983 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:47.201986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:47.202133 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:47.202582 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:47.202586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:47.202588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:47.202966 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:47.203315 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:47.203664 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 02:43:47.203953 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":276,"threads_started":1}
I20260812 02:43:47.206004 27825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:47.292081 27830 heartbeater.cc:503] Master 127.17.196.254:46765 was elected leader, sending a full tablet report...
I20260812 02:43:47.293212 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 27681
W20260812 02:43:47.314412 27437 connection.cc:535] server connection from 127.17.196.194:35383 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:43:47.314610 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 27837
Aug 12 02:43:47 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:43:47.335373 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:47.315922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.17.196.195:46511)
0812 02:43:47.316602 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:47.316613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:47.316651 (+ 38us) server_negotiation.cc:407] Connection header received
0812 02:43:47.316726 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:47.316733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:47.316981 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:47.317219 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:47.318789 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:47.319969 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:47.321668 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:47.322357 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:47.322533 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:47.328061 (+ 5528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:47.328079 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:47.328096 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:47.328144 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:47.332111 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:47.333013 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:47.333022 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:47.333024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:47.333174 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:47.333749 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:47.333754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:47.333756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:47.334209 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:47.334542 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:47.334815 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 02:43:47.335046 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":314,"threads_started":1}
I20260812 02:43:47.336856 27981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:47.423748 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 27837
I20260812 02:43:47.446645 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 27419
I20260812 02:43:47.447926 27492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:43:47.560050 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 27419
I20260812 02:43:47.587993 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46765
--webserver_interface=127.17.196.254
--webserver_port=39181
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:43:47.956092 28007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:47.956430 28007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:47.956594 28007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:47.964243 28007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:43:47.964371 28007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:47.964423 28007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:47.964475 28007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:43:47.964610 28007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:43:47.975098 28007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:46765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=39181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:47.976588 28007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:47.978072 28007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:47.991596 28016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:47.991673 28013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:47.991701 28012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:47.992568 28007 server_base.cc:1034] running on GCE node
I20260812 02:43:47.993367 28007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:47.995085 28007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:47.996315 28007 hybrid_clock.cc:648] HybridClock initialized: now 1786502627996252 us; error 52 us; skew 500 ppm
I20260812 02:43:47.996745 28007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:47 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502627, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:48.005887 28007 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:43:48.007917 28007 webserver.cc:460] Webserver started at http://127.17.196.254:39181/ using document root <none> and password file <none>
I20260812 02:43:48.008709 28007 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:48.008896 28007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:48.015499 28007 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:43:48.019141 28024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:48.020799 28007 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:43:48.020968 28007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "81778cce2a924d17bc4466866555a89d"
format_stamp: "Formatted at 2026-08-12 02:43:44 on dist-test-slave-ss4n"
server_key: "8a41ab0020056bdffb9a7f623e8baef2"
server_key_iv: "da7e65bdfe0c7ef49c4b61896135015a"
server_key_version: "encryptionkey@0"
I20260812 02:43:48.022131 28007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:48.054013 28007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:48.058790 28007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:48.059042 28007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:48.081230 28007 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:46765
I20260812 02:43:48.081244 28085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:46765 every 8 connection(s)
I20260812 02:43:48.082993 28007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:43:48.090767 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 28007
I20260812 02:43:48.090981 28086 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:43:48.091989 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:37291
--local_ip_for_outbound_sockets=127.17.196.193
--tserver_master_addrs=127.17.196.254:46765
--webserver_port=38563
--webserver_interface=127.17.196.193
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.196.193
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:43:48.096628 28086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Bootstrap starting.
I20260812 02:43:48.120534 28086 log.cc:824] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:43:48.131196 28086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:43:48.131865 28086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Bootstrap complete.
I20260812 02:43:48.142472 28086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } }
I20260812 02:43:48.143417 28086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81778cce2a924d17bc4466866555a89d, State: Initialized, Role: FOLLOWER
I20260812 02:43:48.144234 28086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } }
I20260812 02:43:48.144577 28086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:43:48.144719 28086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:43:48.144953 28086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:43:48.149377 28086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } }
I20260812 02:43:48.150079 28086 leader_election.cc:302] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81778cce2a924d17bc4466866555a89d; no voters:
I20260812 02:43:48.150856 28086 leader_election.cc:288] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 02:43:48.151015 28092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:43:48.152578 28092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [term 2 LEADER]: Becoming Leader. State: Replica: 81778cce2a924d17bc4466866555a89d, State: Running, Role: LEADER
I20260812 02:43:48.153393 28092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } }
I20260812 02:43:48.153784 28086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:43:48.156235 28094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81778cce2a924d17bc4466866555a89d. Latest consensus state: current_term: 2 leader_uuid: "81778cce2a924d17bc4466866555a89d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } } }
I20260812 02:43:48.156665 28094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:48.156522 28093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "81778cce2a924d17bc4466866555a89d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81778cce2a924d17bc4466866555a89d" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46765 } } }
I20260812 02:43:48.157014 28093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d [sys.catalog]: This master's current role is: LEADER
I20260812 02:43:48.164413 28099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:43:48.171487 28099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:43:48.173166 28099 catalog_manager.cc:1259] Loaded cluster ID: a0b1786329bf47c4b442c21e5e045079
I20260812 02:43:48.173369 28099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:43:48.177990 28099 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:43:48.179742 28099 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 81778cce2a924d17bc4466866555a89d: Loaded TSK: 0
I20260812 02:43:48.181622 28099 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:43:48.460963 28090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:48.461277 28090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:48.461433 28090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:48.469336 28090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:48.469466 28090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:48.469704 28090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:43:48.480185 28090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.193.keytab
--principal=oryx/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:37291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=38563
--tserver_master_addrs=127.17.196.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:48.481731 28090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:48.483712 28090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:48.499316 28116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:48.499960 28120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:48.500031 28118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:48.501672 28090 server_base.cc:1034] running on GCE node
I20260812 02:43:48.502434 28090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:48.503613 28090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:48.504890 28090 hybrid_clock.cc:648] HybridClock initialized: now 1786502628504812 us; error 71 us; skew 500 ppm
I20260812 02:43:48.505256 28090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:48 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502628, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:48.509258 28090 init.cc:377] Logged in from keytab as oryx/127.17.196.193@KRBTEST.COM (short username oryx)
I20260812 02:43:48.511379 28090 webserver.cc:460] Webserver started at http://127.17.196.193:38563/ using document root <none> and password file <none>
I20260812 02:43:48.512321 28090 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:48.512544 28090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:48.519567 28090 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:43:48.523102 28127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:48.524799 28090 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:43:48.525022 28090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "883261ecd6e74c24bdee9acf394d7016"
format_stamp: "Formatted at 2026-08-12 02:43:45 on dist-test-slave-ss4n"
server_key: "398e48eb6b39fb6dcade2d7dde6df759"
server_key_iv: "c030ffced8129e68caa8df2f6213d7e0"
server_key_version: "encryptionkey@0"
I20260812 02:43:48.526255 28090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:48.547814 28090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:48.553618 28090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:48.553948 28090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:48.555274 28090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:48.556979 28090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:48.557080 28090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:48.557253 28090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:48.557367 28090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:48.598242 28090 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:37291
I20260812 02:43:48.598306 28256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:37291 every 8 connection(s)
I20260812 02:43:48.599937 28090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:43:48.604393 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 28090
I20260812 02:43:48.605374 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:42181
--local_ip_for_outbound_sockets=127.17.196.194
--tserver_master_addrs=127.17.196.254:46765
--webserver_port=37571
--webserver_interface=127.17.196.194
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.196.194
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:43:48 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:48 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:48.625733 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:48.603429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:36529)
0812 02:43:48.605327 (+ 1898us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:48.605400 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:48.608649 (+ 3249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:48.610662 (+ 2013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:48.610677 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:48.611596 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:48.612525 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:48.612559 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.614523 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.614533 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:48.617108 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:48.617132 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.618853 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.618859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:48.619121 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:48.619969 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:48.619995 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:48.624459 (+ 4464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":430,"threads_started":1}
W20260812 02:43:48.626380 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:48.603448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:36529 (local address 127.17.196.254:46765)
0812 02:43:48.605141 (+ 1693us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:48.605161 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:48.605768 (+ 607us) server_negotiation.cc:407] Connection header received
0812 02:43:48.609541 (+ 3773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:48.609579 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:48.610045 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:48.610402 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:48.612843 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.614317 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.617319 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.618581 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.618815 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:48.624964 (+ 6149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:36529: BlockingRecv error: recv got EOF from 127.17.196.193:36529 (error 108)
Metrics: {"server-negotiator.queue_time_us":1360,"thread_start_us":1103,"threads_started":1}
W20260812 02:43:48.626610 28258 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:46765 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.196.254:46765: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Aug 12 02:43:48 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:48 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:48.636709 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:48.627339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:53861)
0812 02:43:48.627630 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:48.627654 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:48.627775 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:48.628210 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:48.628214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:48.628537 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:48.628694 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:48.628702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.630310 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.630314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:48.631484 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:48.631498 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.632611 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.632615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:48.632736 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:48.633382 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:48.633399 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:48.636465 (+ 3066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":53}
W20260812 02:43:48.637071 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:48.627579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:53861 (local address 127.17.196.254:46765)
0812 02:43:48.627779 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:48.627788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:48.627815 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:43:48.627887 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:48.627895 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:48.627985 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:48.628102 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:48.628856 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.630158 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.631657 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.632447 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.632599 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:48.636777 (+ 4178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:53861: BlockingRecv error: recv got EOF from 127.17.196.193:53861 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 02:43:48 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:48 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:48.646680 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:48.637667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:48681)
0812 02:43:48.637842 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:48.637869 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:48.637983 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:48.638371 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:48.638376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:48.638717 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:48.638878 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:48.638887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.640034 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.640041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:48.641158 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:48.641171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.642034 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.642042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:48.642278 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:48.642922 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:48.642941 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:48.646354 (+ 3413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 02:43:48.647058 28258 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:43:48.647281 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:48.637767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:48681 (local address 127.17.196.254:46765)
0812 02:43:48.637974 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:48.637980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:48.638015 (+ 35us) server_negotiation.cc:407] Connection header received
0812 02:43:48.638139 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:48.638149 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:48.638227 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:48.638343 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:48.639047 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.639874 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.641314 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:48.641862 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:48.642016 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:48.646612 (+ 4596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:48681: BlockingRecv error: recv got EOF from 127.17.196.193:48681 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 02:43:48.964545 28262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:48.964850 28262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:48.965013 28262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:48.972565 28262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:48.972692 28262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:48.972810 28262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:43:48.983240 28262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.194.keytab
--principal=oryx/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:42181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=37571
--tserver_master_addrs=127.17.196.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:48.984717 28262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:48.986083 28262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:49.000976 28269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:49.000946 28268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:49.001019 28271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:49.001853 28262 server_base.cc:1034] running on GCE node
I20260812 02:43:49.002626 28262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:49.003749 28262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:49.005021 28262 hybrid_clock.cc:648] HybridClock initialized: now 1786502629004979 us; error 58 us; skew 500 ppm
I20260812 02:43:49.005421 28262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502629, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:49.009325 28262 init.cc:377] Logged in from keytab as oryx/127.17.196.194@KRBTEST.COM (short username oryx)
I20260812 02:43:49.011047 28262 webserver.cc:460] Webserver started at http://127.17.196.194:37571/ using document root <none> and password file <none>
I20260812 02:43:49.011827 28262 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:49.012012 28262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:49.018057 28262 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:43:49.021039 28278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:49.022616 28262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:43:49.022822 28262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "42b3986b8d0642e0ac481b1b70e94c29"
format_stamp: "Formatted at 2026-08-12 02:43:46 on dist-test-slave-ss4n"
server_key: "83c60a466885df6624458f72badd1d45"
server_key_iv: "6a730ef84ea4cee9fee683908bf101a7"
server_key_version: "encryptionkey@0"
I20260812 02:43:49.023852 28262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:49.039397 28262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:49.044344 28262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:49.044670 28262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:49.045841 28262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:49.047333 28262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:49.047468 28262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:49.047617 28262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:49.047711 28262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:49.085722 28262 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:42181
I20260812 02:43:49.085768 28403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:42181 every 8 connection(s)
I20260812 02:43:49.087684 28262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:43:49.090896 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 28262
I20260812 02:43:49.091741 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:46511
--local_ip_for_outbound_sockets=127.17.196.195
--tserver_master_addrs=127.17.196.254:46765
--webserver_port=35829
--webserver_interface=127.17.196.195
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.196.195
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:43:49.108027 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:49.090784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:52089 (local address 127.17.196.254:46765)
0812 02:43:49.090981 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:49.090987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:49.092713 (+ 1726us) server_negotiation.cc:407] Connection header received
0812 02:43:49.095499 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:49.095509 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:49.095622 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:49.095754 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:49.097384 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.098691 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.100404 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.101247 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.101415 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.107757 (+ 6342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:52089: BlockingRecv error: recv got EOF from 127.17.196.194:52089 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:43:49.108819 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:49.091530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:52089)
0812 02:43:49.092514 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:49.092578 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:49.095112 (+ 2534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:49.096012 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:49.096021 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:49.096555 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:49.097166 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.097185 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.098881 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.098886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.100226 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.100240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.101549 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.101558 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.101895 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.102717 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:49.102742 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:49.107516 (+ 4774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":393,"threads_started":1}
W20260812 02:43:49.109697 28404 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:46765 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.196.254:46765: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:49.119879 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:49.110504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:55583)
0812 02:43:49.110765 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:49.110784 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:49.110943 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:49.111342 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:49.111347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:49.111783 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:49.111953 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.111962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.113138 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.113145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.114244 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.114258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.115100 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.115106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.115307 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.116031 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:49.116052 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:49.119568 (+ 3516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105}
W20260812 02:43:49.120424 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:49.110746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:55583 (local address 127.17.196.254:46765)
0812 02:43:49.111014 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:49.111024 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:49.111047 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:43:49.111121 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:49.111125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:49.111190 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:49.111310 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:49.112130 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.112978 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.114432 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.114914 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.115069 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.119763 (+ 4694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:55583: BlockingRecv error: recv got EOF from 127.17.196.194:55583 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:49.131927 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:49.120929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:51439)
0812 02:43:49.121187 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:49.121212 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:49.121368 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:49.121816 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:49.121825 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:49.122373 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:49.122660 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.122678 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.123830 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.123840 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.125779 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.125797 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.126589 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.126596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.126828 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.127525 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:49.127552 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:49.131578 (+ 4026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260812 02:43:49.132139 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:49.120727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:51439 (local address 127.17.196.254:46765)
0812 02:43:49.120889 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:49.120895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:49.121303 (+ 408us) server_negotiation.cc:407] Connection header received
0812 02:43:49.121565 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:49.121572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:49.121652 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:49.121736 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:49.122844 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.123660 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.125948 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.126436 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.126554 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.131860 (+ 5306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:51439: BlockingRecv error: recv got EOF from 127.17.196.194:51439 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 02:43:49.132579 28404 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:43:49.458920 28408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:43:49.459288 28408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:43:49.459461 28408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:43:49.467141 28408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:43:49.467309 28408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:43:49.467497 28408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:43:49.477686 28408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/krb5kdc/oryx_127.17.196.195.keytab
--principal=oryx/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:46511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=35829
--tserver_master_addrs=127.17.196.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:43:49.479157 28408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:43:49.480533 28408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:43:49.495649 28415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:49.495697 28418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:43:49.495697 28416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:43:49.497339 28408 server_base.cc:1034] running on GCE node
I20260812 02:43:49.498219 28408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:43:49.499414 28408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:43:49.500708 28408 hybrid_clock.cc:648] HybridClock initialized: now 1786502629500646 us; error 50 us; skew 500 ppm
I20260812 02:43:49.501089 28408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502629, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:43:49.505184 28408 init.cc:377] Logged in from keytab as oryx/127.17.196.195@KRBTEST.COM (short username oryx)
I20260812 02:43:49.507157 28408 webserver.cc:460] Webserver started at http://127.17.196.195:35829/ using document root <none> and password file <none>
I20260812 02:43:49.507985 28408 fs_manager.cc:359] Metadata directory not provided
I20260812 02:43:49.508167 28408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:43:49.514958 28408 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:43:49.518270 28426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:49.520020 28408 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:43:49.520244 28408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "d6a3a02d3cf340d6af57c5089f7281b0"
format_stamp: "Formatted at 2026-08-12 02:43:46 on dist-test-slave-ss4n"
server_key: "4d8cdd8610a9546893e4cd27b12a27f9"
server_key_iv: "77faf091790dd7ab06d6a4d57b7734cf"
server_key_version: "encryptionkey@0"
I20260812 02:43:49.521299 28408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:43:49.534142 28408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:43:49.538760 28408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:43:49.539115 28408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:43:49.540236 28408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:43:49.541849 28408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:43:49.541990 28408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:49.542176 28408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:43:49.542276 28408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:43:49.582295 28408 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:46511
I20260812 02:43:49.582348 28550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:46511 every 8 connection(s)
I20260812 02:43:49.583900 28408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:43:49.586450 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 28408
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:43:49.607606 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:49.587139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:41837 (local address 127.17.196.254:46765)
0812 02:43:49.587361 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:49.587370 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:49.589360 (+ 1990us) server_negotiation.cc:407] Connection header received
0812 02:43:49.592278 (+ 2918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:49.592284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:49.592397 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:49.592516 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:49.594907 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.596067 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.598960 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.599789 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.599958 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.607298 (+ 7340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:41837: BlockingRecv error: recv got EOF from 127.17.196.195:41837 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:43:49.607947 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:49.587610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40730 (local address 127.17.196.254:46765)
0812 02:43:49.588709 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:49.588721 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:49.588755 (+ 34us) server_negotiation.cc:407] Connection header received
0812 02:43:49.588840 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:49.588846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:49.588957 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:49.589116 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:49.591491 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.592627 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.594644 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.595332 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.595478 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.599148 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:43:49.599157 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:43:49.599174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:43:49.599225 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:43:49.603588 (+ 4363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:49.604561 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:49.604572 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:49.604581 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:49.604847 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:43:49.605374 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:43:49.605379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:43:49.605383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:43:49.606186 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:43:49.606555 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0812 02:43:49.607009 (+ 454us) server_negotiation.cc:299] Negotiation successful
0812 02:43:49.607452 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":733,"threads_started":1}
I20260812 02:43:49.608548 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:49.587877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:41837)
0812 02:43:49.589187 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:49.589242 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:49.591950 (+ 2708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:49.592963 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:49.592977 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:49.593840 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:49.594734 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.594756 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.596310 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.596315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.598727 (+ 2412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.598748 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.600096 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.600107 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.600387 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.601099 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:49.601133 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:49.607034 (+ 5901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":941,"thread_start_us":725,"threads_started":1}
W20260812 02:43:49.609462 28551 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:46765 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.196.254:46765: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:49.619251 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:49.610326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:39777)
0812 02:43:49.610536 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:49.610559 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:49.610723 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:49.611166 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:49.611171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:49.611697 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:49.611901 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.611917 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.613317 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.613321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.614390 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.614405 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.615116 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.615120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.615242 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.615959 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:49.615981 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:49.619039 (+ 3058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 02:43:49.619617 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:49.610528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:39777 (local address 127.17.196.254:46765)
0812 02:43:49.610721 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:49.610725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:49.610742 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:43:49.610871 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:49.610876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:49.611015 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:49.611129 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:49.612219 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.613162 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.614540 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.614965 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.615075 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.619305 (+ 4230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:39777: BlockingRecv error: recv got EOF from 127.17.196.195:39777 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:49.629962 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:49.620142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:36183)
0812 02:43:49.620348 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:49.620375 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:49.620585 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:49.620972 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:49.620977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:49.621337 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:49.621579 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.621591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.622747 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.622756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.624072 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.624087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.625115 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.625121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.625273 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.626050 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:49.626070 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:49.629709 (+ 3639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 02:43:49.630321 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:49.620348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:36183 (local address 127.17.196.254:46765)
0812 02:43:49.620646 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:49.620654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:49.620675 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:43:49.620744 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:49.620747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:49.620810 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:49.620925 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:49.621737 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.622577 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.624223 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.624925 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.625108 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.629979 (+ 4871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:36183: BlockingRecv error: recv got EOF from 127.17.196.195:36183 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
W20260812 02:43:49.630322 28551 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:49 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:49.657979 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:49.647796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:52111)
0812 02:43:49.648484 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:49.648505 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:49.648680 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:49.649183 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:49.649191 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:49.649915 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:49.650218 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.650233 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.651522 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.651528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.652645 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:49.652660 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.653505 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.653516 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:49.653670 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.654323 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:49.654349 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:49.657718 (+ 3369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":351,"threads_started":1}
W20260812 02:43:49.658181 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:49.648037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:52111 (local address 127.17.196.254:46765)
0812 02:43:49.648308 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:49.648316 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:49.648613 (+ 297us) server_negotiation.cc:407] Connection header received
0812 02:43:49.648855 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:49.648860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:49.648956 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:49.649093 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:49.650411 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.651342 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.652815 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:49.653250 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:49.653361 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:49.657946 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:52111: BlockingRecv error: recv got EOF from 127.17.196.193:52111 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 12 02:43:50 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:50 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:50.142446 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:50.133279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:54045)
0812 02:43:50.133877 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:50.133893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:50.134096 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:50.134471 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:50.134476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:50.134990 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:50.135197 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:50.135207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.136282 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.136288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:50.137272 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:50.137284 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.138148 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.138155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:50.138343 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:50.139033 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:50.139052 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:50.142074 (+ 3022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":343,"threads_started":1}
W20260812 02:43:50.142644 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:50.133328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:54045 (local address 127.17.196.254:46765)
0812 02:43:50.133636 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:50.133645 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:50.133993 (+ 348us) server_negotiation.cc:407] Connection header received
0812 02:43:50.134237 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:50.134244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:50.134316 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:50.134424 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:50.135362 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.136129 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.137428 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.137941 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.138095 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:50.142352 (+ 4257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:54045: BlockingRecv error: recv got EOF from 127.17.196.194:54045 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 02:43:50 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:50 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:50.642568 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:50.631008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:41089)
0812 02:43:50.631606 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:50.631624 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:50.631809 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:50.632160 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:50.632163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:50.632696 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:50.632894 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:50.632904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.633967 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.633972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:50.635633 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:50.635651 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.637001 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.637009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:50.637230 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:50.638002 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:50.638023 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:50.642320 (+ 4297us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":327,"threads_started":1}
W20260812 02:43:50.642935 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:50.631108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:41089 (local address 127.17.196.254:46765)
0812 02:43:50.631338 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:50.631343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:50.631748 (+ 405us) server_negotiation.cc:407] Connection header received
0812 02:43:50.631942 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:50.631946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:50.632018 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:50.632112 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:50.633043 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.633817 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.635851 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.636775 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.636923 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:50.642621 (+ 5698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:41089: BlockingRecv error: recv got EOF from 127.17.196.195:41089 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 02:43:50 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:50 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:50.668715 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:50.659064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:50849)
0812 02:43:50.659679 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:50.659698 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:50.659885 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:50.660284 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:50.660290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:50.660755 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:50.660955 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:50.660966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.662211 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.662217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:50.663253 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:50.663268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.664136 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.664143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:50.664340 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:50.665070 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:50.665090 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:50.668393 (+ 3303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":343,"threads_started":1}
W20260812 02:43:50.669162 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:50.659198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:50849 (local address 127.17.196.254:46765)
0812 02:43:50.659434 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:50.659440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:50.659810 (+ 370us) server_negotiation.cc:407] Connection header received
0812 02:43:50.660054 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:50.660059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:50.660129 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:50.660247 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:50.661116 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.662040 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.663423 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:50.663970 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:50.664111 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:50.668714 (+ 4603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:50849: BlockingRecv error: recv got EOF from 127.17.196.193:50849 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 12 02:43:51 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:51 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:51.153914 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:51.143837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:40169)
0812 02:43:51.144510 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:51.144532 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:51.144747 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:51.145182 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:51.145187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:51.145708 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:51.145887 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:51.145897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.146965 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.146970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:51.147883 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:51.147894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.148707 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.148713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:51.148879 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:51.149712 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:51.149736 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:51.153381 (+ 3645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":383,"threads_started":1}
W20260812 02:43:51.154038 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:51.143994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:40169 (local address 127.17.196.254:46765)
0812 02:43:51.144338 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:51.144345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:51.144655 (+ 310us) server_negotiation.cc:407] Connection header received
0812 02:43:51.144888 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:51.144906 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:51.145019 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:51.145143 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:51.146040 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.146799 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.148057 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.148523 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.148665 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:51.153708 (+ 5043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:40169: BlockingRecv error: recv got EOF from 127.17.196.194:40169 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 12 02:43:51 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:51 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:51.654055 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:51.644045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:45751)
0812 02:43:51.644840 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:51.644860 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:51.645042 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:51.645463 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:51.645467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:51.645996 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:51.646206 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:51.646218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.647421 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.647426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:51.648373 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:51.648385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.649209 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.649213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:51.649328 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:51.650095 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:51.650114 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:51.653628 (+ 3514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":485,"threads_started":1}
W20260812 02:43:51.654242 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:51.644215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:45751 (local address 127.17.196.254:46765)
0812 02:43:51.644414 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:51.644419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:51.644963 (+ 544us) server_negotiation.cc:407] Connection header received
0812 02:43:51.645199 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:51.645208 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:51.645297 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:51.645387 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:51.646364 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.647246 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.648592 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.649053 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.649162 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:51.653972 (+ 4810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:45751: BlockingRecv error: recv got EOF from 127.17.196.195:45751 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 02:43:51 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:51 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:51.679445 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:51.669816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:32815)
0812 02:43:51.670467 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:51.670485 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:51.670626 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:51.670948 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:51.670951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:51.671332 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:51.671594 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:51.671607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.672902 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.672907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:51.674042 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:51.674057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.674936 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.674942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:51.675135 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:51.675831 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:51.675852 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:51.679154 (+ 3302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":400,"threads_started":1}
W20260812 02:43:51.679704 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:51.669899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:32815 (local address 127.17.196.254:46765)
0812 02:43:51.670099 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:51.670111 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:51.670610 (+ 499us) server_negotiation.cc:407] Connection header received
0812 02:43:51.670738 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:51.670742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:51.670810 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:51.670891 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:51.671797 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.672737 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.674229 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:51.674739 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:51.674860 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:51.679453 (+ 4593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:32815: BlockingRecv error: recv got EOF from 127.17.196.193:32815 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 02:43:52 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:52 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:52.164100 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:52.155102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:56819)
0812 02:43:52.155686 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:52.155714 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:52.155843 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:52.156195 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:52.156198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:52.156545 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:52.156723 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:52.156731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.157884 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.157889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:52.158819 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:52.158832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.159731 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.159736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:52.159889 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:52.160620 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:52.160639 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:52.163697 (+ 3058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":359,"threads_started":1}
W20260812 02:43:52.164276 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:52.155193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:56819 (local address 127.17.196.254:46765)
0812 02:43:52.155376 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:52.155381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:52.155825 (+ 444us) server_negotiation.cc:407] Connection header received
0812 02:43:52.155967 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:52.155971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:52.156059 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:52.156154 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:52.156881 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.157719 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.158984 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.159420 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.159519 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:52.163935 (+ 4416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:56819: BlockingRecv error: recv got EOF from 127.17.196.194:56819 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 02:43:52 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:52 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:52.666167 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:52.655389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:50447)
0812 02:43:52.656123 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:52.656156 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:52.656359 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:52.656838 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:52.656844 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:52.657309 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:52.657560 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:52.657572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.658819 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.658828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:52.659931 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:52.659947 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.660938 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.660946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:52.661193 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:52.662040 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:52.662063 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:52.665821 (+ 3758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":332,"threads_started":1}
W20260812 02:43:52.666450 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:52.655597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:50447 (local address 127.17.196.254:46765)
0812 02:43:52.655882 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:52.655890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:52.656269 (+ 379us) server_negotiation.cc:407] Connection header received
0812 02:43:52.656582 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:52.656587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:52.656661 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:52.656795 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:52.657751 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.658632 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.660184 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.660737 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.660921 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:52.666178 (+ 5257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:50447: BlockingRecv error: recv got EOF from 127.17.196.195:50447 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 12 02:43:52 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:52 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:52.689927 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:52.680554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:55087)
0812 02:43:52.681150 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:52.681167 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:52.681296 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:52.681798 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:52.681803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:52.682246 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:52.682437 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:52.682448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.683638 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.683642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:52.684795 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:52.684810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.685689 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.685694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:52.685825 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:52.686577 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:52.686596 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:52.689687 (+ 3091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":335,"threads_started":1}
W20260812 02:43:52.690327 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:52.680782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:55087 (local address 127.17.196.254:46765)
0812 02:43:52.681000 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:52.681005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:52.681284 (+ 279us) server_negotiation.cc:407] Connection header received
0812 02:43:52.681413 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:52.681417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:52.681612 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:52.681712 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:52.682605 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.683430 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.684946 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:52.685362 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:52.685544 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:52.689931 (+ 4387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:55087: BlockingRecv error: recv got EOF from 127.17.196.193:55087 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 12 02:43:53 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:53 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:53.181545 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:53.165520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:55721)
0812 02:43:53.166112 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:53.166130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:53.166258 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:53.166835 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:53.166840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:53.167248 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:53.167409 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:53.167419 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.168735 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.168740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:53.170136 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:53.170152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.171230 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.171237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:53.171409 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:53.177925 (+ 6516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:53.177957 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:53.181236 (+ 3279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":323,"threads_started":1}
W20260812 02:43:53.181932 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:53.165595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:55721 (local address 127.17.196.254:46765)
0812 02:43:53.165779 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:53.165784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:53.166227 (+ 443us) server_negotiation.cc:407] Connection header received
0812 02:43:53.166446 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:53.166454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:53.166602 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:53.166715 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:53.167614 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.168417 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.170326 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.171034 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.171160 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:53.181648 (+ 10488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:55721: BlockingRecv error: recv got EOF from 127.17.196.194:55721 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 02:43:53 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:53 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:53.677054 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:53.667423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:39307)
0812 02:43:53.668097 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:53.668117 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:53.668260 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:53.668730 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:53.668734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:53.669167 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:53.669332 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:53.669342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.670556 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.670562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:53.671830 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:53.671850 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.672752 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.672759 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:53.672955 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:53.673615 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:53.673635 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:53.676792 (+ 3157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":341,"threads_started":1}
W20260812 02:43:53.677301 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:53.667515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:39307 (local address 127.17.196.254:46765)
0812 02:43:53.667745 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:53.667750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:53.668250 (+ 500us) server_negotiation.cc:407] Connection header received
0812 02:43:53.668393 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:53.668397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:53.668527 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:53.668624 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:53.669630 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.670361 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.671989 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.672582 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.672737 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:53.677043 (+ 4306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:39307: BlockingRecv error: recv got EOF from 127.17.196.195:39307 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 12 02:43:53 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:53 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:53.701222 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:53.691145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:35919)
0812 02:43:53.691774 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:53.691793 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:53.691931 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:53.692318 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:53.692322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:53.692808 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:53.693010 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:53.693020 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.694206 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.694211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:53.695300 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:53.695315 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.696240 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.696246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:53.696470 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:53.697203 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:53.697224 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:53.700927 (+ 3703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":322,"threads_started":1}
W20260812 02:43:53.701525 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:53.691213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:35919 (local address 127.17.196.254:46765)
0812 02:43:53.691550 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:53.691561 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:53.691918 (+ 357us) server_negotiation.cc:407] Connection header received
0812 02:43:53.692074 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:53.692080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:53.692158 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:53.692250 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:53.693165 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.694028 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.695449 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:53.696057 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:53.696207 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:53.701263 (+ 5056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:35919: BlockingRecv error: recv got EOF from 127.17.196.193:35919 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 02:43:54 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:54 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:54.192422 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:54.182761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:56587)
0812 02:43:54.183388 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:54.183407 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:54.183594 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:54.184081 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:54.184085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:54.184650 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:54.184889 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:54.184901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.186259 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.186267 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:54.187408 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:54.187423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.188220 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.188225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:54.188365 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:54.189021 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:54.189041 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:54.192149 (+ 3108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":284,"threads_started":1}
W20260812 02:43:54.192775 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:54.183078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:56587 (local address 127.17.196.254:46765)
0812 02:43:54.183325 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:54.183334 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:54.183504 (+ 170us) server_negotiation.cc:407] Connection header received
0812 02:43:54.183762 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:54.183767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:54.183851 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:54.184029 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:54.185101 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.186098 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.187571 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.188077 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.188187 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:54.192389 (+ 4202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:56587: BlockingRecv error: recv got EOF from 127.17.196.194:56587 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 12 02:43:54 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:54 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:54.688678 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:54.678297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:49013)
0812 02:43:54.679006 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:54.679024 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:54.679190 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:54.679740 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:54.679753 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:54.680253 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:54.680463 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:54.680475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.681836 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.681841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:54.682900 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:54.682915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.683809 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.683814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:54.683965 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:54.684725 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:54.684746 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:54.688258 (+ 3512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":411,"threads_started":1}
W20260812 02:43:54.688817 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:54.678419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:49013 (local address 127.17.196.254:46765)
0812 02:43:54.678633 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:54.678639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:54.679144 (+ 505us) server_negotiation.cc:407] Connection header received
0812 02:43:54.679363 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:54.679368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:54.679456 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:54.679573 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:54.680657 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.681638 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.683098 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.683629 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.683758 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:54.688529 (+ 4771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:49013: BlockingRecv error: recv got EOF from 127.17.196.195:49013 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 02:43:54 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:54 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:54.711400 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:54.702477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:33057)
0812 02:43:54.703040 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:54.703057 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:54.703236 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:54.703576 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:54.703581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:54.704038 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:54.704195 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:54.704204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.705251 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.705258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:54.706258 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:54.706270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.707055 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.707061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:54.707249 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:54.707937 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:54.707954 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:54.711144 (+ 3190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":314,"threads_started":1}
W20260812 02:43:54.711804 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:54.702600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:33057 (local address 127.17.196.254:46765)
0812 02:43:54.702832 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:54.702841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:54.703158 (+ 317us) server_negotiation.cc:407] Connection header received
0812 02:43:54.703373 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:54.703377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:54.703438 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:54.703543 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:54.704345 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.705078 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.706430 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:54.706884 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:54.707030 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:54.711445 (+ 4415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:33057: BlockingRecv error: recv got EOF from 127.17.196.193:33057 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 02:43:55 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:55 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:55.202065 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:55.193351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:53971)
0812 02:43:55.193911 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:55.193928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:55.194059 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:55.194373 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:55.194376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:55.194766 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:55.194935 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:55.194943 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.195921 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.195925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:55.196875 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:55.196889 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.197748 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.197752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:55.197881 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:55.198600 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:55.198617 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:55.201807 (+ 3190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":323,"threads_started":1}
W20260812 02:43:55.202235 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:55.193583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:53971 (local address 127.17.196.254:46765)
0812 02:43:55.193837 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:55.193849 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:55.194015 (+ 166us) server_negotiation.cc:407] Connection header received
0812 02:43:55.194163 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:55.194167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:55.194244 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:55.194328 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:55.195065 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.195779 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.197059 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.197597 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.197711 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:55.202015 (+ 4304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:53971: BlockingRecv error: recv got EOF from 127.17.196.194:53971 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 02:43:55 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:55 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:55.700017 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:55.689946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:33281)
0812 02:43:55.690722 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:55.690744 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:55.690881 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:55.691233 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:55.691237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:55.691768 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:55.691930 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:55.691940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.693239 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.693244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:55.694469 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:55.694487 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.695648 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.695654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:55.695814 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:55.696646 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:55.696666 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:55.699801 (+ 3135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":328,"threads_started":1}
W20260812 02:43:55.700266 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:55.690200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:33281 (local address 127.17.196.254:46765)
0812 02:43:55.690446 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:55.690453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:55.690848 (+ 395us) server_negotiation.cc:407] Connection header received
0812 02:43:55.691025 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:55.691029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:55.691095 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:55.691180 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:55.692127 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.693050 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.694829 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.695319 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.695532 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:55.700013 (+ 4481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:33281: BlockingRecv error: recv got EOF from 127.17.196.195:33281 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 12 02:43:55 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:55 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:55.722425 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:55.712506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:46561)
0812 02:43:55.713195 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:55.713214 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:55.713356 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:55.713833 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:55.713837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:55.714340 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:55.714531 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:55.714545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.715743 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.715748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:55.716880 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:55.716896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.717986 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.717992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:55.718140 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:55.718843 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:55.718865 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:55.722120 (+ 3255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":309,"threads_started":1}
W20260812 02:43:55.722802 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:55.712608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:46561 (local address 127.17.196.254:46765)
0812 02:43:55.712872 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:55.712883 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:55.713335 (+ 452us) server_negotiation.cc:407] Connection header received
0812 02:43:55.713456 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:55.713461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:55.713670 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:55.713767 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:55.714706 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.715532 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.717063 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:55.717788 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:55.717911 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:55.722526 (+ 4615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:46561: BlockingRecv error: recv got EOF from 127.17.196.193:46561 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 02:43:56 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:56 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:56.213328 28577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:56.203360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:60821)
0812 02:43:56.204041 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:56.204059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:56.204190 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:56.204693 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:56.204700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:56.205300 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:56.205569 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:56.205582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.207082 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.207086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:56.208074 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:56.208086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.208860 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.208864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:56.208984 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:56.209693 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:56.209713 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:56.213071 (+ 3358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":330,"threads_started":1}
W20260812 02:43:56.213593 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:56.203476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:60821 (local address 127.17.196.254:46765)
0812 02:43:56.203828 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:56.203837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:56.204176 (+ 339us) server_negotiation.cc:407] Connection header received
0812 02:43:56.204302 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:56.204308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:56.204424 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:56.204615 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:56.205737 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.206920 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.208238 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.208705 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.208808 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:56.213276 (+ 4468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:60821: BlockingRecv error: recv got EOF from 127.17.196.194:60821 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Aug 12 02:43:56 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:56 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:56.711053 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:56.701263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:41657)
0812 02:43:56.701967 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:56.701984 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:56.702125 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:56.702465 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:56.702468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:56.702926 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:56.703115 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:56.703124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.704276 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.704280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:56.705239 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:56.705252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.706166 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.706172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:56.706357 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:56.707095 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:56.707122 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:56.710769 (+ 3647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":422,"threads_started":1}
W20260812 02:43:56.711443 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:56.701383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:41657 (local address 127.17.196.254:46765)
0812 02:43:56.701704 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:56.701716 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:56.702094 (+ 378us) server_negotiation.cc:407] Connection header received
0812 02:43:56.702257 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:56.702262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:56.702331 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:56.702436 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:56.703265 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.704115 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.705407 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.705938 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.706106 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:56.711163 (+ 5057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:41657: BlockingRecv error: recv got EOF from 127.17.196.195:41657 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Aug 12 02:43:56 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:56 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:56.734297 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:56.723568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:37483)
0812 02:43:56.724223 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:56.724244 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:56.724440 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:56.725065 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:56.725071 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:56.725609 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:56.725775 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:56.725785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.727073 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.727088 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:56.728107 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:56.728121 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.729201 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.729206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:56.729324 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:56.730095 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:56.730122 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:56.733876 (+ 3754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":332,"threads_started":1}
W20260812 02:43:56.734484 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:56.723748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:37483 (local address 127.17.196.254:46765)
0812 02:43:56.723956 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:56.723961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:56.724372 (+ 411us) server_negotiation.cc:407] Connection header received
0812 02:43:56.724622 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:56.724629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:56.724718 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:56.724794 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:56.725931 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.726833 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.728286 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:56.728996 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:56.729121 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:56.734149 (+ 5028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:37483: BlockingRecv error: recv got EOF from 127.17.196.193:37483 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 02:43:57 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:57 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:57.225459 28581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:57.214424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:42839)
0812 02:43:57.215119 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:57.215138 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:57.215270 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:57.215703 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:57.215712 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:57.216287 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:57.216519 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:57.216537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.218115 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.218120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:57.219288 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:57.219308 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.220403 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.220409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:57.220551 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:57.221378 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:57.221399 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:57.225036 (+ 3637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":385,"threads_started":1}
W20260812 02:43:57.225612 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:57.214554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:42839 (local address 127.17.196.254:46765)
0812 02:43:57.214775 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:57.214781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:57.215224 (+ 443us) server_negotiation.cc:407] Connection header received
0812 02:43:57.215384 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:57.215389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:57.215459 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:57.215561 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:57.216765 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.217885 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.219469 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.220150 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.220280 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:57.225280 (+ 5000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:42839: BlockingRecv error: recv got EOF from 127.17.196.194:42839 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 02:43:57 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:57 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:57.722677 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:57.712435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:39907)
0812 02:43:57.713023 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:57.713041 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:57.713167 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:57.713552 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:57.713558 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:57.714016 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:57.714181 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:57.714190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.715538 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.715547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:57.716636 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:57.716650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.717473 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.717544 (+ 71us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:57.717707 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:57.718563 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:57.718585 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:57.722288 (+ 3703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":314,"threads_started":1}
W20260812 02:43:57.722961 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:57.712522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:39907 (local address 127.17.196.254:46765)
0812 02:43:57.712840 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:57.712848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:57.713185 (+ 337us) server_negotiation.cc:407] Connection header received
0812 02:43:57.713268 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:57.713273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:57.713344 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:57.713440 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:57.714383 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.715279 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.716825 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.717246 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.717354 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:57.722624 (+ 5270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:39907: BlockingRecv error: recv got EOF from 127.17.196.195:39907 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 12 02:43:57 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:57 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:57.746188 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:57.735690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:32945)
0812 02:43:57.736874 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:57.736893 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:57.737083 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:57.737607 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:57.737616 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:57.738164 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:57.738348 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:57.738359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.739602 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.739607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:57.740723 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:57.740738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.741649 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.741654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:57.741800 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:57.742470 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:57.742545 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:57.745924 (+ 3379us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1060,"thread_start_us":896,"threads_started":1}
W20260812 02:43:57.746708 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:57.735722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:32945 (local address 127.17.196.254:46765)
0812 02:43:57.735947 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:57.735954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:57.736996 (+ 1042us) server_negotiation.cc:407] Connection header received
0812 02:43:57.737225 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:57.737230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:57.737307 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:57.737427 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:57.738571 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.739409 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.740914 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:57.741396 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:57.741647 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:57.746207 (+ 4560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:32945: BlockingRecv error: recv got EOF from 127.17.196.193:32945 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 02:43:58 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:58 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:58.243445 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:58.227059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:35899)
0812 02:43:58.227753 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:58.227772 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:58.227940 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:58.228383 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:58.228388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:58.228993 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:58.229195 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:58.229206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.230458 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.230487 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:58.231739 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:58.231756 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.232622 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.232628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:58.232770 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:58.239800 (+ 7030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:58.239824 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:58.243192 (+ 3368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":330,"threads_started":1}
W20260812 02:43:58.243788 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:58.227071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:35899 (local address 127.17.196.254:46765)
0812 02:43:58.227324 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:58.227333 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:58.227865 (+ 532us) server_negotiation.cc:407] Connection header received
0812 02:43:58.228139 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:58.228145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:58.228227 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:58.228345 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:58.229364 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.230298 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.231925 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.232415 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.232556 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:58.243451 (+ 10895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:35899: BlockingRecv error: recv got EOF from 127.17.196.194:35899 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 02:43:58 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:58 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:58.734411 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:58.723983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:56761)
0812 02:43:58.724634 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:58.724655 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:58.724857 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:58.725279 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:58.725283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:58.725962 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:58.726150 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:58.726160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.727249 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.727255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:58.728429 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:58.728446 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.729698 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.729704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:58.729915 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:58.730672 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:58.730696 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:58.734134 (+ 3438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":369,"threads_started":1}
W20260812 02:43:58.734688 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:58.724312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:56761 (local address 127.17.196.254:46765)
0812 02:43:58.724637 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:58.724645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:58.724775 (+ 130us) server_negotiation.cc:407] Connection header received
0812 02:43:58.725055 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:58.725059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:58.725126 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:58.725246 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:58.726305 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.727094 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.728676 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.729439 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.729682 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:58.734373 (+ 4691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:56761: BlockingRecv error: recv got EOF from 127.17.196.195:56761 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 12 02:43:58 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:58 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:58.757992 28587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:58.747359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:37465)
0812 02:43:58.748063 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:58.748081 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:58.748273 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:58.748759 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:58.748764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:58.749252 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:58.749419 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:58.749430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.750694 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.750703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:58.751789 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:58.751803 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.752660 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.752667 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:58.752846 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:58.753701 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:58.753721 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:58.757384 (+ 3663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":410,"threads_started":1}
W20260812 02:43:58.757988 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:58.747571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:37465 (local address 127.17.196.254:46765)
0812 02:43:58.747866 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:58.747876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:58.748183 (+ 307us) server_negotiation.cc:407] Connection header received
0812 02:43:58.748422 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:58.748428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:58.748584 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:58.748715 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:58.749635 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.750523 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.751961 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:58.752446 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:58.752610 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:58.757711 (+ 5101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:37465: BlockingRecv error: recv got EOF from 127.17.196.193:37465 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 12 02:43:59 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:59 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:59.253834 28588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:59.244500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:40247)
0812 02:43:59.245355 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:59.245379 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:59.245570 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:59.245968 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:59.245974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:59.246421 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:59.246632 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:59.246642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.247689 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.247693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:59.248647 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:59.248658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.249413 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.249417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:59.249646 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:59.250349 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:59.250368 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:59.253561 (+ 3193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":310,"threads_started":1}
W20260812 02:43:59.254034 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:59.244675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:40247 (local address 127.17.196.254:46765)
0812 02:43:59.244916 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:59.244922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:59.245524 (+ 602us) server_negotiation.cc:407] Connection header received
0812 02:43:59.245711 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:59.245716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:59.245797 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:59.245918 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:59.246774 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.247549 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.248794 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.249263 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.249376 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:59.253796 (+ 4420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:40247: BlockingRecv error: recv got EOF from 127.17.196.194:40247 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 02:43:59 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:59 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:59.746246 28589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:59.735755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:56205)
0812 02:43:59.736362 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:59.736383 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:59.736566 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:59.737035 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:59.737040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:59.737673 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:59.737927 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:59.737943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.739071 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.739078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:59.740192 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:59.740208 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.741161 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.741168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:59.741365 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:59.742156 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:59.742177 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:59.745866 (+ 3689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":329,"threads_started":1}
W20260812 02:43:59.746475 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:59.736055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:56205 (local address 127.17.196.254:46765)
0812 02:43:59.736367 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:59.736377 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:59.736504 (+ 127us) server_negotiation.cc:407] Connection header received
0812 02:43:59.736716 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:59.736725 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:59.736838 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:59.736972 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:59.738102 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.738910 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.740379 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.740953 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.741129 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:59.746163 (+ 5034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:56205: BlockingRecv error: recv got EOF from 127.17.196.195:56205 (error 108)
Metrics: {"server-negotiator.queue_time_us":170}
Aug 12 02:43:59 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:43:59 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:43:59.768826 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:43:59.759193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:49703)
0812 02:43:59.759947 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0812 02:43:59.759965 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:43:59.760101 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:43:59.760525 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:43:59.760528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:43:59.761014 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:43:59.761268 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:59.761282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.762468 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.762475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:59.763396 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:43:59.763408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.764255 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.764261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:43:59.764450 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:59.765204 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:43:59.765223 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:43:59.768470 (+ 3247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":483,"threads_started":1}
W20260812 02:43:59.768975 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:43:59.759269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:49703 (local address 127.17.196.254:46765)
0812 02:43:59.759498 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 02:43:59.759504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:43:59.760071 (+ 567us) server_negotiation.cc:407] Connection header received
0812 02:43:59.760293 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:43:59.760298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:43:59.760367 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:43:59.760472 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:43:59.761430 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.762316 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.763571 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:43:59.764093 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:43:59.764232 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:43:59.768706 (+ 4474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:49703: BlockingRecv error: recv got EOF from 127.17.196.193:49703 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 12 02:44:00 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:00 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:00.264396 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:00.255171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:46253)
0812 02:44:00.255755 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:00.255773 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:00.255956 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:00.256403 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:00.256407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:00.256993 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:00.257262 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:00.257277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.258440 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.258444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:00.259342 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:00.259353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.260013 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.260018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:00.260176 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:00.260842 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:00.260860 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:00.264148 (+ 3288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":337,"threads_started":1}
W20260812 02:44:00.264945 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:00.255285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:46253 (local address 127.17.196.254:46765)
0812 02:44:00.255500 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:00.255508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:00.255886 (+ 378us) server_negotiation.cc:407] Connection header received
0812 02:44:00.256125 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:00.256131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:00.256223 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:00.256349 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:00.257447 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.258309 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.259478 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.259875 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.259970 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:00.264560 (+ 4590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:46253: BlockingRecv error: recv got EOF from 127.17.196.194:46253 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 02:44:00 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:00 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:00.758033 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:00.747624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:54843)
0812 02:44:00.748312 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:00.748331 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:00.748548 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:00.748915 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:00.748918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:00.749383 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:00.749693 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:00.749705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.750993 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.751000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:00.752138 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:00.752156 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.753202 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.753209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:00.753446 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:00.754303 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:00.754323 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:00.757742 (+ 3419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":402,"threads_started":1}
W20260812 02:44:00.758549 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:00.747818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:54843 (local address 127.17.196.254:46765)
0812 02:44:00.748068 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:00.748073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:00.748568 (+ 495us) server_negotiation.cc:407] Connection header received
0812 02:44:00.748655 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:00.748660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:00.748731 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:00.748845 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:00.749857 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.750797 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.752370 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.752979 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.753119 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:00.758024 (+ 4905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:54843: BlockingRecv error: recv got EOF from 127.17.196.195:54843 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 02:44:00 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:00 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:00.780719 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:00.770286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:57329)
0812 02:44:00.770804 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:00.770821 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:00.770952 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:00.771300 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:00.771304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:00.771765 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:00.771928 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:00.771937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.773212 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.773217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:00.774536 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:00.774553 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.775428 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.775434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:00.775612 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:00.776344 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:00.776364 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:00.780313 (+ 3949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":306,"threads_started":1}
W20260812 02:44:00.780908 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:00.770373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:57329 (local address 127.17.196.254:46765)
0812 02:44:00.770655 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:00.770663 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:00.770912 (+ 249us) server_negotiation.cc:407] Connection header received
0812 02:44:00.771069 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:00.771073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:00.771143 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:00.771236 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:00.772141 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.773023 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.774721 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:00.775252 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:00.775370 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:00.780557 (+ 5187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:57329: BlockingRecv error: recv got EOF from 127.17.196.193:57329 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 12 02:44:01 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:01 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:01.276021 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:01.265689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:32895)
0812 02:44:01.266334 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:01.266353 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:01.266537 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:01.266966 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:01.266971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:01.267509 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:01.267750 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:01.267766 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.269313 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.269318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:01.270434 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:01.270449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.271254 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.271259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:01.271407 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:01.272204 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:01.272225 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:01.275635 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":388,"threads_started":1}
W20260812 02:44:01.276190 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:01.265805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:32895 (local address 127.17.196.254:46765)
0812 02:44:01.266008 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:01.266014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:01.266485 (+ 471us) server_negotiation.cc:407] Connection header received
0812 02:44:01.266703 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:01.266709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:01.266797 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:01.266941 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:01.267928 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.269127 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.270621 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.271096 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.271205 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:01.275899 (+ 4694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:32895: BlockingRecv error: recv got EOF from 127.17.196.194:32895 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 02:44:01 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:01 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:01.769254 28597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:01.759217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:36441)
0812 02:44:01.759838 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:01.759856 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:01.759983 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:01.760382 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:01.760385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:01.760865 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:01.761034 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:01.761044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.762235 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.762241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:01.763391 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:01.763407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.764333 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.764338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:01.764521 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:01.765209 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:01.765234 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:01.768862 (+ 3628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":367,"threads_started":1}
W20260812 02:44:01.769460 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:01.759262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:36441 (local address 127.17.196.254:46765)
0812 02:44:01.759558 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:01.759563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:01.759970 (+ 407us) server_negotiation.cc:407] Connection header received
0812 02:44:01.760145 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:01.760150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:01.760224 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:01.760316 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:01.761204 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.762045 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.763588 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.764090 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.764223 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:01.769114 (+ 4891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:36441: BlockingRecv error: recv got EOF from 127.17.196.195:36441 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 02:44:01 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:01 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:01.792237 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:01.782072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:47581)
0812 02:44:01.782714 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:01.782737 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:01.782902 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:01.783346 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:01.783377 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:01.783960 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:01.784184 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:01.784195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.785551 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.785565 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:01.786607 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:01.786623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.787504 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.787510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:01.787662 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:01.788520 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:01.788544 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:01.791964 (+ 3420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":306,"threads_started":1}
W20260812 02:44:01.792590 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:01.782400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:47581 (local address 127.17.196.254:46765)
0812 02:44:01.782618 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:01.782624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:01.782832 (+ 208us) server_negotiation.cc:407] Connection header received
0812 02:44:01.783069 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:01.783073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:01.783138 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:01.783288 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:01.784343 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.785302 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.786785 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:01.787274 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:01.787472 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:01.792219 (+ 4747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:47581: BlockingRecv error: recv got EOF from 127.17.196.193:47581 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 02:44:02 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:02 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:02.286790 28599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:02.277299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:56017)
0812 02:44:02.278053 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:02.278075 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:02.278250 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:02.278637 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:02.278645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:02.279110 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:02.279289 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:02.279298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.280411 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.280419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:02.281378 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:02.281391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.282234 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.282241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:02.282426 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:02.283130 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:02.283149 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:02.286532 (+ 3383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":468,"threads_started":1}
W20260812 02:44:02.287163 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:02.277464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:56017 (local address 127.17.196.254:46765)
0812 02:44:02.277707 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:02.277715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:02.278181 (+ 466us) server_negotiation.cc:407] Connection header received
0812 02:44:02.278392 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:02.278398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:02.278479 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:02.278591 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:02.279449 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.280246 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.281562 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.282015 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.282181 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:02.286857 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:56017: BlockingRecv error: recv got EOF from 127.17.196.194:56017 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 12 02:44:02 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:02 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:02.780351 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:02.770367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:41935)
0812 02:44:02.770993 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:02.771010 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:02.771183 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:02.771627 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:02.771633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:02.772099 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:02.772264 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:02.772272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.773352 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.773357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:02.774420 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:02.774434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.775245 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.775251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:02.775423 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:02.776147 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:02.776169 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:02.779945 (+ 3776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":337,"threads_started":1}
W20260812 02:44:02.780572 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:02.770477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:41935 (local address 127.17.196.254:46765)
0812 02:44:02.770800 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:02.770809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:02.771116 (+ 307us) server_negotiation.cc:407] Connection header received
0812 02:44:02.771309 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:02.771317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:02.771388 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:02.771499 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:02.772430 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.773203 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.774590 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.775039 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.775200 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:02.780236 (+ 5036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:41935: BlockingRecv error: recv got EOF from 127.17.196.195:41935 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 12 02:44:02 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:02 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:02.802943 28601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:02.793328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:48693)
0812 02:44:02.793927 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:02.793953 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:02.794096 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:02.794479 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:02.794484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:02.795105 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:02.795333 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:02.795343 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.796395 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.796400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:02.797433 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:02.797448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.798211 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.798216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:02.798351 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:02.799235 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:02.799255 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:02.802667 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":314,"threads_started":1}
W20260812 02:44:02.803342 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:02.793397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:48693 (local address 127.17.196.254:46765)
0812 02:44:02.793807 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:02.793813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:02.794056 (+ 243us) server_negotiation.cc:407] Connection header received
0812 02:44:02.794217 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:02.794223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:02.794305 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:02.794398 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:02.795471 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.796224 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.797599 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:02.798052 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:02.798150 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:02.802884 (+ 4734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:48693: BlockingRecv error: recv got EOF from 127.17.196.193:48693 (error 108)
Metrics: {"server-negotiator.queue_time_us":243}
Aug 12 02:44:03 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:03 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:03.303628 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:03.287944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:59825)
0812 02:44:03.288534 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:03.288553 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:03.288684 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:03.289067 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:03.289071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:03.289533 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:03.289759 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:03.289770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.291028 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.291033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:03.292060 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:03.292074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.293400 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.293405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:03.293661 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:03.299546 (+ 5885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:03.299571 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:03.303204 (+ 3633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":328,"threads_started":1}
W20260812 02:44:03.303742 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:03.288057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:59825 (local address 127.17.196.254:46765)
0812 02:44:03.288297 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:03.288302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:03.288665 (+ 363us) server_negotiation.cc:407] Connection header received
0812 02:44:03.288832 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:03.288837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:03.288909 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:03.289036 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:03.289946 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.290832 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.292233 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.292940 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.293124 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:03.303476 (+ 10352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:59825: BlockingRecv error: recv got EOF from 127.17.196.194:59825 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 02:44:03 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:03 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.195@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:03.791785 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:03.781984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.195:52971)
0812 02:44:03.782620 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:03.782637 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:03.782820 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:03.783193 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:03.783198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:03.783603 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:03.783799 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:03.783808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.785057 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.785063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:03.786460 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:03.786475 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.787472 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.787477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:03.787654 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:03.788420 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:03.788440 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:03.791527 (+ 3087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":355,"threads_started":1}
W20260812 02:44:03.792192 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:03.782174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:52971 (local address 127.17.196.254:46765)
0812 02:44:03.782528 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:03.782540 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:03.782738 (+ 198us) server_negotiation.cc:407] Connection header received
0812 02:44:03.782970 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:03.782976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:03.783043 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:03.783159 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:03.783970 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.784894 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.786653 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.787292 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.787416 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:03.791890 (+ 4474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.195:52971: BlockingRecv error: recv got EOF from 127.17.196.195:52971 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 12 02:44:03 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:03 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.193@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:44:03.814014 28605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:03.804300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.193:43649)
0812 02:44:03.804866 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:03.804884 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:03.805007 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:03.805378 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:03.805381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:03.805936 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:03.806146 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:03.806157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.807350 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.807354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:03.808377 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:03.808389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.809200 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.809205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:03.809341 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:03.810326 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:03.810356 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:03.813725 (+ 3369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":320,"threads_started":1}
W20260812 02:44:03.814190 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:03.804356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:43649 (local address 127.17.196.254:46765)
0812 02:44:03.804612 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:03.804622 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:03.804974 (+ 352us) server_negotiation.cc:407] Connection header received
0812 02:44:03.805162 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:03.805166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:03.805242 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:03.805335 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:03.806322 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.807206 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.808546 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:03.809058 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:03.809163 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:03.813943 (+ 4780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.193:43649: BlockingRecv error: recv got EOF from 127.17.196.193:43649 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 02:44:04 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:44:04 dist-test-slave-ss4n krb5kdc[27404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.196.194@KRBTEST.COM for oryx/127.17.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:44:04.315342 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:04.305318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:44287 (local address 127.17.196.254:46765)
0812 02:44:04.305618 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:04.305626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:04.306045 (+ 419us) server_negotiation.cc:407] Connection header received
0812 02:44:04.306269 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:04.306273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:04.306344 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:04.306452 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:04.307333 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.308130 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.309412 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.309915 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.310059 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:04.314948 (+ 4889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.196.194:44287: BlockingRecv error: recv got EOF from 127.17.196.194:44287 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 02:44:04.315757 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:04.305165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46765 (local address 127.17.196.194:44287)
0812 02:44:04.305924 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:04.305941 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:04.306115 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:04.306486 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:04.306490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:04.306986 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:04.307172 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:04.307181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.308283 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.308289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:04.309259 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:04.309272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.310077 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.310083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:04.310276 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:04.310918 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:04.310937 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:04.315417 (+ 4480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.196.254:46765: Server oryx/127.17.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":476,"threads_started":1}
I20260812 02:44:04.593194 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28090
I20260812 02:44:04.608847 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:04.594484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.17.196.193:37291)
0812 02:44:04.595181 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:04.595189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:04.595217 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:44:04.595280 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:04.595287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:04.595504 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:04.595686 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:04.597571 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.598821 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.600910 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.601654 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.601770 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:04.605218 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:04.605228 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:04.605245 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:04.605290 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:04.608294 (+ 3004us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.196.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 02:44:04.608609 (+ 315us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51060: Request ticket server kudu/127.17.196.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":325,"threads_started":1}
W20260812 02:44:04.608990 28608 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51060: Request ticket server kudu/127.17.196.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 02:44:04.609179 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28090: Not authorized: Client connection negotiation failed: client connection to 127.17.196.193:37291: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.196.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 02:44:04.609416 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 28090
I20260812 02:44:04.634775 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28262
I20260812 02:44:04.649853 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:04.636168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.17.196.194:42181)
0812 02:44:04.637036 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:04.637045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:04.637081 (+ 36us) server_negotiation.cc:407] Connection header received
0812 02:44:04.637150 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:04.637157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:04.637400 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:04.637651 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:04.639276 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.640602 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.642217 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.642897 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.643013 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:04.645978 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:04.645989 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:04.646005 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:04.646067 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:04.649110 (+ 3043us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.196.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 02:44:04.649414 (+ 304us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53856: Request ticket server kudu/127.17.196.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":473,"threads_started":1}
W20260812 02:44:04.649991 28609 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53856: Request ticket server kudu/127.17.196.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 02:44:04.650046 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28262: Not authorized: Client connection negotiation failed: client connection to 127.17.196.194:42181: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.196.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 02:44:04.650233 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 28262
I20260812 02:44:04.678776 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28408
I20260812 02:44:04.693214 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:04.680168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.17.196.195:46511)
0812 02:44:04.680931 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:04.680939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:04.680969 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:44:04.681053 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:04.681060 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:04.681287 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:04.681448 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:04.683131 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.684373 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.685872 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:04.686521 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:04.686682 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:04.689643 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:04.689654 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:04.689670 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:04.689711 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:04.692620 (+ 2909us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.196.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 02:44:04.692952 (+ 332us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36642: Request ticket server kudu/127.17.196.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":457,"threads_started":1}
W20260812 02:44:04.693377 28610 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36642: Request ticket server kudu/127.17.196.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 02:44:04.693562 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28408: Not authorized: Client connection negotiation failed: client connection to 127.17.196.195:46511: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.196.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 02:44:04.693781 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 28408
I20260812 02:44:04.720024 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28007
I20260812 02:44:04.721827 28081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:04.850442 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 28007
2026-08-12T02:44:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22020 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T02:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:44:04Z Disabled control of system clock
I20260812 02:44:04.955675 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:43709
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:43709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 02:44:05.326932 28616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:05.327270 28616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:05.327353 28616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:05.334831 28616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:44:05.334967 28616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:05.335005 28616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:05.335062 28616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:44:05.335115 28616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:44:05.345021 28616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:43709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:43709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:05.346671 28616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:05.349601 28616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:05.371023 28621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:05.371050 28622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:05.371438 28616 server_base.cc:1034] running on GCE node
W20260812 02:44:05.371250 28624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:05.372987 28616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:05.375433 28616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:05.376771 28616 hybrid_clock.cc:648] HybridClock initialized: now 1786502645376698 us; error 63 us; skew 500 ppm
I20260812 02:44:05.377277 28616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:44:05.381012 28616 webserver.cc:460] Webserver started at http://127.17.196.254:44899/ using document root <none> and password file <none>
I20260812 02:44:05.382071 28616 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:05.382246 28616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:05.382762 28616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:05.387563 28616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "ac727070d4cf4b9ba2e5be398735d211"
format_stamp: "Formatted at 2026-08-12 02:44:05 on dist-test-slave-ss4n"
server_key: "81fc1ab434b114a7afd73e29563c7d91"
server_key_iv: "daa3e53d7631b3ec1dd94a6fc018e774"
server_key_version: "encryptionkey@0"
I20260812 02:44:05.388814 28616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "ac727070d4cf4b9ba2e5be398735d211"
format_stamp: "Formatted at 2026-08-12 02:44:05 on dist-test-slave-ss4n"
server_key: "81fc1ab434b114a7afd73e29563c7d91"
server_key_iv: "daa3e53d7631b3ec1dd94a6fc018e774"
server_key_version: "encryptionkey@0"
I20260812 02:44:05.397231 28616 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260812 02:44:05.403092 28631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:05.405335 28616 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260812 02:44:05.405822 28616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "ac727070d4cf4b9ba2e5be398735d211"
format_stamp: "Formatted at 2026-08-12 02:44:05 on dist-test-slave-ss4n"
server_key: "81fc1ab434b114a7afd73e29563c7d91"
server_key_iv: "daa3e53d7631b3ec1dd94a6fc018e774"
server_key_version: "encryptionkey@0"
I20260812 02:44:05.406204 28616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:05.427269 28616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:05.428371 28616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:05.428716 28616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:05.450358 28616 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:43709
I20260812 02:44:05.450373 28690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:43709 every 8 connection(s)
I20260812 02:44:05.451959 28616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:44:05.456342 28691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:44:05.459052 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 28616
I20260812 02:44:05.459290 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:44:05.460181 18195 external_mini_cluster.cc:1428] Setting key abd6309e1e9b3e8d85fd14037c1657bb
I20260812 02:44:05.470515 28691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211: Bootstrap starting.
I20260812 02:44:05.474572 28691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211: Neither blocks nor log segments found. Creating new log.
I20260812 02:44:05.475914 28691 log.cc:824] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211: Log is configured to *not* fsync() on all Append() calls
I20260812 02:44:05.476516 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:05.463081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.17.196.254:43709)
0812 02:44:05.464215 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:05.464228 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:05.464277 (+ 49us) server_negotiation.cc:407] Connection header received
0812 02:44:05.466205 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:05.466233 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:05.466796 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:05.467185 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:05.468031 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:05.469415 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:05.471424 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:05.472181 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:05.472343 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:05.473814 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:05.473822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:05.473837 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:05.474444 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:05.474574 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:05.474600 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:05.474762 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:05.474993 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 02:44:05.475351 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":430,"threads_started":1}
I20260812 02:44:05.479601 28691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211: No bootstrap required, opened a new log
I20260812 02:44:05.487882 28691 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac727070d4cf4b9ba2e5be398735d211" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43709 } }
I20260812 02:44:05.488309 28691 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:44:05.488415 28691 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac727070d4cf4b9ba2e5be398735d211, State: Initialized, Role: FOLLOWER
I20260812 02:44:05.489115 28691 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac727070d4cf4b9ba2e5be398735d211" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43709 } }
I20260812 02:44:05.489343 28691 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:44:05.489439 28691 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:44:05.489652 28691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:44:05.492331 28691 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac727070d4cf4b9ba2e5be398735d211" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43709 } }
I20260812 02:44:05.492966 28691 leader_election.cc:302] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac727070d4cf4b9ba2e5be398735d211; no voters:
I20260812 02:44:05.493737 28691 leader_election.cc:288] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:44:05.493921 28697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:44:05.494747 28697 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [term 1 LEADER]: Becoming Leader. State: Replica: ac727070d4cf4b9ba2e5be398735d211, State: Running, Role: LEADER
I20260812 02:44:05.495404 28697 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac727070d4cf4b9ba2e5be398735d211" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43709 } }
I20260812 02:44:05.496057 28691 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:44:05.498929 28699 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac727070d4cf4b9ba2e5be398735d211" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac727070d4cf4b9ba2e5be398735d211" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43709 } } }
I20260812 02:44:05.499423 28699 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [sys.catalog]: This master's current role is: LEADER
I20260812 02:44:05.499400 28700 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac727070d4cf4b9ba2e5be398735d211. Latest consensus state: current_term: 1 leader_uuid: "ac727070d4cf4b9ba2e5be398735d211" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac727070d4cf4b9ba2e5be398735d211" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43709 } } }
I20260812 02:44:05.499765 28700 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211 [sys.catalog]: This master's current role is: LEADER
I20260812 02:44:05.502806 28705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:44:05.509297 28705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:44:05.521137 28705 catalog_manager.cc:1347] Generated new cluster ID: e055c92c1fe44e5da1dc2aaad1622bde
I20260812 02:44:05.521275 28705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:44:05.533303 28705 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:44:05.534785 28705 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:44:05.546114 28705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac727070d4cf4b9ba2e5be398735d211: Generated new TSK 0
I20260812 02:44:05.547350 28705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:44:05.554148 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43709
--builtin_ntp_servers=127.17.196.212:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 02:44:05.922663 28718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:05.923106 28718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:05.923275 28718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:05.931267 28718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:05.931440 28718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:05.931630 28718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:44:05.941457 28718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:05.942983 28718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:05.944988 28718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:05.959704 28727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:05.959676 28725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:05.959738 28723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:05.960489 28718 server_base.cc:1034] running on GCE node
I20260812 02:44:05.961169 28718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:05.962337 28718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:05.963655 28718 hybrid_clock.cc:648] HybridClock initialized: now 1786502645963589 us; error 76 us; skew 500 ppm
I20260812 02:44:05.964008 28718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:44:05.966609 28718 webserver.cc:460] Webserver started at http://127.17.196.193:42323/ using document root <none> and password file <none>
I20260812 02:44:05.967351 28718 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:05.967497 28718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:05.967849 28718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:05.970871 28718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "276629106f0d401f97222710155647f7"
format_stamp: "Formatted at 2026-08-12 02:44:05 on dist-test-slave-ss4n"
server_key: "bf4e45547c2bc4921128fad689ff5c37"
server_key_iv: "a25a1da6ab18f902f83904154fc43009"
server_key_version: "encryptionkey@0"
I20260812 02:44:05.971725 28718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "276629106f0d401f97222710155647f7"
format_stamp: "Formatted at 2026-08-12 02:44:05 on dist-test-slave-ss4n"
server_key: "bf4e45547c2bc4921128fad689ff5c37"
server_key_iv: "a25a1da6ab18f902f83904154fc43009"
server_key_version: "encryptionkey@0"
I20260812 02:44:05.977267 28718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:44:05.981014 28733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:05.982576 28718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:44:05.982771 28718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "276629106f0d401f97222710155647f7"
format_stamp: "Formatted at 2026-08-12 02:44:05 on dist-test-slave-ss4n"
server_key: "bf4e45547c2bc4921128fad689ff5c37"
server_key_iv: "a25a1da6ab18f902f83904154fc43009"
server_key_version: "encryptionkey@0"
I20260812 02:44:05.983001 28718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:06.004639 28718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:06.005738 28718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:06.006093 28718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:06.007217 28718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:06.008819 28718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:06.008952 28718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:06.009089 28718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:06.009178 28718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:06.047144 28718 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:41595
I20260812 02:44:06.047245 28862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:41595 every 8 connection(s)
I20260812 02:44:06.048658 28718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:44:06.056753 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 28718
I20260812 02:44:06.057057 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:44:06.057821 18195 external_mini_cluster.cc:1428] Setting key 95646f7e5601eeb83b02d0fca3d5761d
I20260812 02:44:06.060647 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43709
--builtin_ntp_servers=127.17.196.212:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 02:44:06.062304 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:06.051576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:38945 (local address 127.17.196.254:43709)
0812 02:44:06.052060 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:06.052065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:06.053464 (+ 1399us) server_negotiation.cc:407] Connection header received
0812 02:44:06.055333 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:06.055338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:06.055433 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:06.055569 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:06.056685 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:06.057579 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:06.059243 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:06.059664 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:06.059804 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:06.061380 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:06.061385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:06.061388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:06.061406 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:06.061447 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:06.061460 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:06.061592 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:06.061825 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 02:44:06.061932 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":239,"threads_started":1}
I20260812 02:44:06.063418 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:06.052311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:43709 (local address 127.17.196.193:38945)
0812 02:44:06.053276 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:06.053316 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:06.055045 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:06.055815 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:06.055824 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:06.055914 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:06.056507 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:06.056525 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:06.057755 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:06.057759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:06.059080 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:06.059093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:06.059920 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:06.059930 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:06.060190 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:06.061028 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:44:06.061031 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:06.061076 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:44:06.061167 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:06.061590 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:06.061602 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:06.061613 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 02:44:06.061813 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 02:44:06.062293 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":287,"threads_started":1}
I20260812 02:44:06.065213 28863 heartbeater.cc:347] Connected to a master server at 127.17.196.254:43709
I20260812 02:44:06.065593 28863 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:06.066398 28863 heartbeater.cc:511] Master 127.17.196.254:43709 requested a full tablet report, sending...
I20260812 02:44:06.068974 28651 ts_manager.cc:194] Registered new tserver with Master: 276629106f0d401f97222710155647f7 (127.17.196.193:41595)
I20260812 02:44:06.071267 28651 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.193:38945
W20260812 02:44:06.416671 28870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:06.417034 28870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:06.417205 28870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:06.424667 28870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:06.424831 28870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:06.425009 28870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:44:06.434620 28870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:06.436043 28870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:06.438028 28870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:06.451836 28876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:06.451969 28875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:06.452670 28878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:06.453805 28870 server_base.cc:1034] running on GCE node
I20260812 02:44:06.454483 28870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:06.455577 28870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:06.456838 28870 hybrid_clock.cc:648] HybridClock initialized: now 1786502646456769 us; error 57 us; skew 500 ppm
I20260812 02:44:06.457204 28870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:44:06.459759 28870 webserver.cc:460] Webserver started at http://127.17.196.194:39673/ using document root <none> and password file <none>
I20260812 02:44:06.460515 28870 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:06.460659 28870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:06.460997 28870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:06.464121 28870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "5044f9be79b840e4badcd7c329ace494"
format_stamp: "Formatted at 2026-08-12 02:44:06 on dist-test-slave-ss4n"
server_key: "b972959d7f263e36c3f3ede63f3ceb1b"
server_key_iv: "7c658bd3a8adee6b8fc56bd0205ea441"
server_key_version: "encryptionkey@0"
I20260812 02:44:06.464978 28870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "5044f9be79b840e4badcd7c329ace494"
format_stamp: "Formatted at 2026-08-12 02:44:06 on dist-test-slave-ss4n"
server_key: "b972959d7f263e36c3f3ede63f3ceb1b"
server_key_iv: "7c658bd3a8adee6b8fc56bd0205ea441"
server_key_version: "encryptionkey@0"
I20260812 02:44:06.470763 28870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:44:06.474778 28884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:06.476477 28870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260812 02:44:06.476662 28870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "5044f9be79b840e4badcd7c329ace494"
format_stamp: "Formatted at 2026-08-12 02:44:06 on dist-test-slave-ss4n"
server_key: "b972959d7f263e36c3f3ede63f3ceb1b"
server_key_iv: "7c658bd3a8adee6b8fc56bd0205ea441"
server_key_version: "encryptionkey@0"
I20260812 02:44:06.476879 28870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:06.514204 28870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:06.515082 28870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:06.515359 28870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:06.516340 28870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:06.517897 28870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:06.517990 28870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:06.518075 28870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:06.518131 28870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:06.557081 28870 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:36073
I20260812 02:44:06.557181 29010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:36073 every 8 connection(s)
I20260812 02:44:06.558548 28870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:44:06.564556 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 28870
I20260812 02:44:06.564765 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:44:06.565250 18195 external_mini_cluster.cc:1428] Setting key 9358bfb7550c141ce9d9c7cc1516c131
I20260812 02:44:06.568346 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43709
--builtin_ntp_servers=127.17.196.212:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 02:44:06.572929 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:06.561256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:49375 (local address 127.17.196.254:43709)
0812 02:44:06.561539 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:06.561546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:06.563026 (+ 1480us) server_negotiation.cc:407] Connection header received
0812 02:44:06.565151 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:06.565157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:06.565252 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:06.565363 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:06.566533 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:06.567272 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:06.569808 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:06.570212 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:06.570313 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:06.572133 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:06.572139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:06.572143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:06.572169 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:06.572204 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:06.572218 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:06.572299 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:06.572606 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 02:44:06.572691 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:44:06.574528 29014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:06.561900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:43709 (local address 127.17.196.194:49375)
0812 02:44:06.562866 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:06.562902 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:06.564836 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:06.565656 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:06.565665 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:06.565756 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:06.566359 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:06.566377 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:06.567439 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:06.567444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:06.569636 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:06.569669 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:06.570457 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:06.570466 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:06.570841 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:06.571643 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:44:06.571649 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:06.571708 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:44:06.571817 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:06.572327 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:06.572340 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:06.572351 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 02:44:06.572540 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 02:44:06.572992 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":298,"threads_started":1}
I20260812 02:44:06.576238 29011 heartbeater.cc:347] Connected to a master server at 127.17.196.254:43709
I20260812 02:44:06.576588 29011 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:06.577677 29011 heartbeater.cc:511] Master 127.17.196.254:43709 requested a full tablet report, sending...
I20260812 02:44:06.579368 28651 ts_manager.cc:194] Registered new tserver with Master: 5044f9be79b840e4badcd7c329ace494 (127.17.196.194:36073)
I20260812 02:44:06.580608 28651 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.194:49375
W20260812 02:44:06.926285 29015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:06.926682 29015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:06.926859 29015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:06.934547 29015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:06.934722 29015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:06.934911 29015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:44:06.944525 29015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:06.945998 29015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:06.947958 29015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:06.962589 29023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:06.962883 29021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:06.962623 29020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:06.963521 29015 server_base.cc:1034] running on GCE node
I20260812 02:44:06.964184 29015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:06.965232 29015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:06.966490 29015 hybrid_clock.cc:648] HybridClock initialized: now 1786502646966466 us; error 106 us; skew 500 ppm
I20260812 02:44:06.966853 29015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:44:06.969605 29015 webserver.cc:460] Webserver started at http://127.17.196.195:44375/ using document root <none> and password file <none>
I20260812 02:44:06.970418 29015 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:06.970558 29015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:06.970868 29015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:06.973944 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "342ec16910e94b7ba1ffc99577426aec"
format_stamp: "Formatted at 2026-08-12 02:44:06 on dist-test-slave-ss4n"
server_key: "d4dc43bb89b6b14088fcbdb5e2b15c4e"
server_key_iv: "078961a36a60030d84c3b40d0a3f0fbd"
server_key_version: "encryptionkey@0"
I20260812 02:44:06.974741 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "342ec16910e94b7ba1ffc99577426aec"
format_stamp: "Formatted at 2026-08-12 02:44:06 on dist-test-slave-ss4n"
server_key: "d4dc43bb89b6b14088fcbdb5e2b15c4e"
server_key_iv: "078961a36a60030d84c3b40d0a3f0fbd"
server_key_version: "encryptionkey@0"
I20260812 02:44:06.980266 29015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:44:06.984398 29030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:06.986114 29015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:44:06.986277 29015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "342ec16910e94b7ba1ffc99577426aec"
format_stamp: "Formatted at 2026-08-12 02:44:06 on dist-test-slave-ss4n"
server_key: "d4dc43bb89b6b14088fcbdb5e2b15c4e"
server_key_iv: "078961a36a60030d84c3b40d0a3f0fbd"
server_key_version: "encryptionkey@0"
I20260812 02:44:06.986490 29015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:07.009027 29015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:07.010197 29015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:07.010651 29015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:07.012106 29015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:07.014055 29015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:07.014199 29015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:07.014374 29015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:07.014477 29015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:07.056342 29015 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:38025
I20260812 02:44:07.056450 29155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:38025 every 8 connection(s)
I20260812 02:44:07.057879 29015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:44:07.062884 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 29015
I20260812 02:44:07.063092 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:44:07.063673 18195 external_mini_cluster.cc:1428] Setting key fef66991a39c9b6aa2d6979fc89b7664
I20260812 02:44:07.072346 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:07.061072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:44723 (local address 127.17.196.254:43709)
0812 02:44:07.061326 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:07.061331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:07.063235 (+ 1904us) server_negotiation.cc:407] Connection header received
0812 02:44:07.065156 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:07.065162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:07.065274 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:07.065393 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:07.066636 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.067426 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.069135 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.069628 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.069774 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:07.071364 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:07.071369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:07.071372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:07.071393 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:07.071426 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:07.071438 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:07.071541 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:07.071789 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 02:44:07.071907 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260812 02:44:07.073573 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:07.061725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:43709 (local address 127.17.196.195:44723)
0812 02:44:07.062820 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:07.062866 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:07.064875 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:07.065695 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:07.065704 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:07.065817 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:07.066444 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:07.066463 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.067620 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.067624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:07.068938 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:07.068951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.069885 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.069893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:07.070167 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:07.070986 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:44:07.070988 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:07.071034 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:44:07.071133 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:07.071541 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:07.071562 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:07.071572 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 02:44:07.071778 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 02:44:07.072231 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":360,"threads_started":1}
I20260812 02:44:07.075505 29156 heartbeater.cc:347] Connected to a master server at 127.17.196.254:43709
I20260812 02:44:07.075618 28863 heartbeater.cc:503] Master 127.17.196.254:43709 was elected leader, sending a full tablet report...
I20260812 02:44:07.075872 29156 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:07.076761 29156 heartbeater.cc:511] Master 127.17.196.254:43709 requested a full tablet report, sending...
I20260812 02:44:07.078449 28651 ts_manager.cc:194] Registered new tserver with Master: 342ec16910e94b7ba1ffc99577426aec (127.17.196.195:38025)
I20260812 02:44:07.079634 28651 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.195:44723
I20260812 02:44:07.080101 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 02:44:07.087209 28868 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:07.085231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.17.196.254:43709)
0812 02:44:07.085423 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:07.085429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:07.086129 (+ 700us) server_negotiation.cc:407] Connection header received
0812 02:44:07.086272 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:07.086277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:07.086361 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:07.086461 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:07.086982 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35708: BlockingRecv error: recv got EOF from 127.0.0.1:35708 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:44:07.088037 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28718
I20260812 02:44:07.097153 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:07.089197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.17.196.193:41595)
0812 02:44:07.089953 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:07.089960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:07.089986 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:44:07.090043 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:07.090051 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:07.090270 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:07.090407 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:07.090874 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.092029 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.093473 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.094127 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.094229 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:07.095370 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:07.095380 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:07.095395 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:07.096041 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:07.096095 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:07.096119 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:07.096232 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:07.096594 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 02:44:07.096780 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":455,"threads_started":1}
I20260812 02:44:07.098662 28858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:07.186072 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 28718
I20260812 02:44:07.208285 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28870
I20260812 02:44:07.217824 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:07.209703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53834 (local address 127.17.196.194:36073)
0812 02:44:07.210429 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:07.210442 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:07.210486 (+ 44us) server_negotiation.cc:407] Connection header received
0812 02:44:07.210640 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:07.210652 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:07.210912 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:07.211074 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:07.211594 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.212804 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.214127 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.214737 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.214843 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:07.216023 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:07.216032 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:07.216047 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:07.216744 (+ 697us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:07.216797 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:07.216819 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:07.216919 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:07.217153 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 02:44:07.217319 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":332,"threads_started":1}
I20260812 02:44:07.219331 29006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:07.306826 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 28870
I20260812 02:44:07.328819 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 29015
I20260812 02:44:07.338558 29170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:07.330143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.17.196.195:38025)
0812 02:44:07.330822 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:07.330830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:07.330873 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:44:07.330943 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:07.330949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:07.331178 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:07.331321 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:07.331803 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.332891 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.334424 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:07.335030 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:07.335138 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:07.336399 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:07.336482 (+ 83us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:07.336499 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:07.337214 (+ 715us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:07.337302 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:07.337333 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:07.337468 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:07.337841 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 02:44:07.338034 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":340,"threads_started":1}
I20260812 02:44:07.340143 29150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:07.429000 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 29015
I20260812 02:44:07.451548 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 28616
I20260812 02:44:07.452983 28686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:07.558481 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 28616
2026-08-12T02:44:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2701 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T02:44:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:44:07Z Disabled control of system clock
I20260812 02:44:07.652154 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:43359
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:43359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 02:44:08.007253 29176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:08.007675 29176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:08.007838 29176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:08.015427 29176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:44:08.015621 29176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:08.015722 29176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:08.015822 29176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:44:08.015940 29176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:44:08.026168 29176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:43359
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:43359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:08.027480 29176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:08.029309 29176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:08.042537 29183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:08.042481 29182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:08.042883 29185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:08.043494 29176 server_base.cc:1034] running on GCE node
I20260812 02:44:08.044569 29176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:08.046281 29176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:08.047586 29176 hybrid_clock.cc:648] HybridClock initialized: now 1786502648047398 us; error 118 us; skew 500 ppm
I20260812 02:44:08.048017 29176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:44:08.050839 29176 webserver.cc:460] Webserver started at http://127.17.196.254:35763/ using document root <none> and password file <none>
I20260812 02:44:08.051630 29176 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:08.051726 29176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:08.052134 29176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:08.055284 29176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "28827e97c7ed4a2ebecbfd961aa49908"
format_stamp: "Formatted at 2026-08-12 02:44:08 on dist-test-slave-ss4n"
server_key: "c92147b861fbaa6e8370fd18b3310418"
server_key_iv: "201b0563be86381dbd2baff70c9d97a9"
server_key_version: "encryptionkey@0"
I20260812 02:44:08.056154 29176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "28827e97c7ed4a2ebecbfd961aa49908"
format_stamp: "Formatted at 2026-08-12 02:44:08 on dist-test-slave-ss4n"
server_key: "c92147b861fbaa6e8370fd18b3310418"
server_key_iv: "201b0563be86381dbd2baff70c9d97a9"
server_key_version: "encryptionkey@0"
I20260812 02:44:08.061713 29176 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:44:08.065757 29194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:08.067514 29176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:44:08.067780 29176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "28827e97c7ed4a2ebecbfd961aa49908"
format_stamp: "Formatted at 2026-08-12 02:44:08 on dist-test-slave-ss4n"
server_key: "c92147b861fbaa6e8370fd18b3310418"
server_key_iv: "201b0563be86381dbd2baff70c9d97a9"
server_key_version: "encryptionkey@0"
I20260812 02:44:08.068070 29176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:08.075253 29176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:08.076285 29176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:08.076649 29176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:08.099030 29176 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:43359
I20260812 02:44:08.099076 29253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:43359 every 8 connection(s)
I20260812 02:44:08.100569 29176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:44:08.102162 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 29176
I20260812 02:44:08.102435 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:44:08.103440 18195 external_mini_cluster.cc:1428] Setting key e30b6d924bd18044a95ad732991b2e32
I20260812 02:44:08.105967 29254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:44:08.113204 29257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:08.105904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35786 (local address 127.17.196.254:43359)
0812 02:44:08.107233 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:08.107246 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:08.107291 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:44:08.108998 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:08.109021 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:08.109472 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:08.110002 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:08.111242 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:08.111248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:08.111262 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:08.111278 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:08.111363 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:08.111394 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:08.111687 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:08.111854 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 02:44:08.112148 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":553,"threads_started":1}
I20260812 02:44:08.119705 29254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908: Bootstrap starting.
I20260812 02:44:08.123271 29254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908: Neither blocks nor log segments found. Creating new log.
I20260812 02:44:08.124423 29254 log.cc:824] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908: Log is configured to *not* fsync() on all Append() calls
I20260812 02:44:08.127779 29254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908: No bootstrap required, opened a new log
I20260812 02:44:08.136196 29254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28827e97c7ed4a2ebecbfd961aa49908" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43359 } }
I20260812 02:44:08.136632 29254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:44:08.136750 29254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28827e97c7ed4a2ebecbfd961aa49908, State: Initialized, Role: FOLLOWER
I20260812 02:44:08.137578 29254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28827e97c7ed4a2ebecbfd961aa49908" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43359 } }
I20260812 02:44:08.137890 29254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:44:08.138083 29254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:44:08.138310 29254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:44:08.140825 29254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28827e97c7ed4a2ebecbfd961aa49908" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43359 } }
I20260812 02:44:08.141458 29254 leader_election.cc:302] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28827e97c7ed4a2ebecbfd961aa49908; no voters:
I20260812 02:44:08.142225 29254 leader_election.cc:288] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:44:08.142403 29259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:44:08.143358 29259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [term 1 LEADER]: Becoming Leader. State: Replica: 28827e97c7ed4a2ebecbfd961aa49908, State: Running, Role: LEADER
I20260812 02:44:08.144114 29259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28827e97c7ed4a2ebecbfd961aa49908" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43359 } }
I20260812 02:44:08.144989 29254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:44:08.147701 29260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28827e97c7ed4a2ebecbfd961aa49908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28827e97c7ed4a2ebecbfd961aa49908" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43359 } } }
I20260812 02:44:08.147701 29261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28827e97c7ed4a2ebecbfd961aa49908. Latest consensus state: current_term: 1 leader_uuid: "28827e97c7ed4a2ebecbfd961aa49908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28827e97c7ed4a2ebecbfd961aa49908" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43359 } } }
I20260812 02:44:08.148185 29260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [sys.catalog]: This master's current role is: LEADER
I20260812 02:44:08.148274 29261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908 [sys.catalog]: This master's current role is: LEADER
I20260812 02:44:08.150240 29266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:44:08.155184 29266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:44:08.168076 29266 catalog_manager.cc:1347] Generated new cluster ID: 2b04a8bbc5f8437e96927ae0c40bce9a
I20260812 02:44:08.168237 29266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:44:08.188683 29266 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:44:08.189194 29266 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:44:08.199756 29266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28827e97c7ed4a2ebecbfd961aa49908: Generated new TSK 0
I20260812 02:44:08.200816 29266 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:44:08.218952 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43359
--builtin_ntp_servers=127.17.196.212:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 02:44:08.584262 29279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:08.584587 29279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:08.584749 29279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:08.592358 29279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:08.592489 29279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:08.592674 29279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:44:08.602399 29279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:08.603842 29279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:08.605772 29279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:08.620184 29288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:08.620204 29284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:08.620093 29285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:08.621608 29279 server_base.cc:1034] running on GCE node
I20260812 02:44:08.622328 29279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:08.623380 29279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:08.624629 29279 hybrid_clock.cc:648] HybridClock initialized: now 1786502648624590 us; error 39 us; skew 500 ppm
I20260812 02:44:08.624994 29279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:44:08.631803 29279 webserver.cc:460] Webserver started at http://127.17.196.193:43759/ using document root <none> and password file <none>
I20260812 02:44:08.632584 29279 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:08.632741 29279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:08.633085 29279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:08.636231 29279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "418c694f7f7a4106a8024b5222c9c239"
format_stamp: "Formatted at 2026-08-12 02:44:08 on dist-test-slave-ss4n"
server_key: "bbff0f5af6904366d6077d482796489b"
server_key_iv: "a8b06216d672ef561aaffdece7895975"
server_key_version: "encryptionkey@0"
I20260812 02:44:08.637061 29279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "418c694f7f7a4106a8024b5222c9c239"
format_stamp: "Formatted at 2026-08-12 02:44:08 on dist-test-slave-ss4n"
server_key: "bbff0f5af6904366d6077d482796489b"
server_key_iv: "a8b06216d672ef561aaffdece7895975"
server_key_version: "encryptionkey@0"
I20260812 02:44:08.642885 29279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:44:08.646919 29294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:08.648537 29279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:44:08.648773 29279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "418c694f7f7a4106a8024b5222c9c239"
format_stamp: "Formatted at 2026-08-12 02:44:08 on dist-test-slave-ss4n"
server_key: "bbff0f5af6904366d6077d482796489b"
server_key_iv: "a8b06216d672ef561aaffdece7895975"
server_key_version: "encryptionkey@0"
I20260812 02:44:08.649086 29279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:08.655521 29279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:08.656383 29279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:08.656672 29279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:08.657835 29279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:08.659464 29279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:08.659561 29279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:08.659731 29279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:08.659816 29279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:08.696796 29279 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:45783
I20260812 02:44:08.696986 29418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:45783 every 8 connection(s)
I20260812 02:44:08.698312 29279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:44:08.703276 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 29279
I20260812 02:44:08.703537 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:44:08.704418 18195 external_mini_cluster.cc:1428] Setting key 91d52570dcba694cfc2d57620dbc62b1
I20260812 02:44:08.708027 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43359
--builtin_ntp_servers=127.17.196.212:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 02:44:08.708638 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:08.701291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:45827 (local address 127.17.196.254:43359)
0812 02:44:08.701805 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:08.701812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:08.703609 (+ 1797us) server_negotiation.cc:407] Connection header received
0812 02:44:08.706033 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:08.706038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:08.706101 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:08.706230 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:08.707775 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:08.707781 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:08.707789 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:08.707816 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:08.707874 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:08.707892 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:08.708055 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:08.708238 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 02:44:08.708313 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":261,"threads_started":1}
I20260812 02:44:08.709774 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:08.702023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:43359 (local address 127.17.196.193:45827)
0812 02:44:08.703338 (+ 1315us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:08.703393 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:08.705676 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:08.706449 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:08.706458 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:08.706561 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:08.707423 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:44:08.707426 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:08.707472 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:44:08.707582 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:08.708020 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:08.708028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:08.708035 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 02:44:08.708328 (+ 293us) client_negotiation.cc:240] Negotiation successful
0812 02:44:08.708628 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":296,"threads_started":1}
I20260812 02:44:08.711935 29419 heartbeater.cc:347] Connected to a master server at 127.17.196.254:43359
I20260812 02:44:08.712304 29419 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:08.713019 29419 heartbeater.cc:511] Master 127.17.196.254:43359 requested a full tablet report, sending...
I20260812 02:44:08.715541 29214 ts_manager.cc:194] Registered new tserver with Master: 418c694f7f7a4106a8024b5222c9c239 (127.17.196.193:45783)
W20260812 02:44:09.084955 29423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:09.085338 29423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:09.085568 29423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:09.093948 29423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:09.094118 29423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:09.094331 29423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:44:09.105289 29423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:09.106868 29423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:09.108865 29423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:09.124286 29428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:09.124401 29432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:09.124500 29429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:09.125409 29423 server_base.cc:1034] running on GCE node
I20260812 02:44:09.126292 29423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:09.127403 29423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:09.128628 29423 hybrid_clock.cc:648] HybridClock initialized: now 1786502649128567 us; error 43 us; skew 500 ppm
I20260812 02:44:09.129066 29423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:44:09.131876 29423 webserver.cc:460] Webserver started at http://127.17.196.194:36595/ using document root <none> and password file <none>
I20260812 02:44:09.132658 29423 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:09.132787 29423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:09.133167 29423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:09.136551 29423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "411418ef10ce495cbd309099b1e3d591"
format_stamp: "Formatted at 2026-08-12 02:44:09 on dist-test-slave-ss4n"
server_key: "03df8efe5f560b029bd6eb7700ec1aaa"
server_key_iv: "f980c8af99586853d6218c626dd2d119"
server_key_version: "encryptionkey@0"
I20260812 02:44:09.137548 29423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "411418ef10ce495cbd309099b1e3d591"
format_stamp: "Formatted at 2026-08-12 02:44:09 on dist-test-slave-ss4n"
server_key: "03df8efe5f560b029bd6eb7700ec1aaa"
server_key_iv: "f980c8af99586853d6218c626dd2d119"
server_key_version: "encryptionkey@0"
I20260812 02:44:09.143487 29423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:44:09.147691 29441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:09.149302 29423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:44:09.149492 29423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "411418ef10ce495cbd309099b1e3d591"
format_stamp: "Formatted at 2026-08-12 02:44:09 on dist-test-slave-ss4n"
server_key: "03df8efe5f560b029bd6eb7700ec1aaa"
server_key_iv: "f980c8af99586853d6218c626dd2d119"
server_key_version: "encryptionkey@0"
I20260812 02:44:09.149796 29423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:09.156700 29423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:09.157657 29423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:09.157966 29423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:09.159089 29423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:09.160722 29423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:09.160862 29423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:09.161020 29423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:09.161139 29423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:09.199187 29423 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:35913
I20260812 02:44:09.199288 29568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:35913 every 8 connection(s)
I20260812 02:44:09.200709 29423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:44:09.201934 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 29423
I20260812 02:44:09.202195 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:44:09.202833 18195 external_mini_cluster.cc:1428] Setting key 29f5a4d4757c2128b1fcc15d2ac63080
I20260812 02:44:09.206470 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43359
--builtin_ntp_servers=127.17.196.212:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 02:44:09.210611 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:09.204256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:39519 (local address 127.17.196.254:43359)
0812 02:44:09.204540 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:09.204547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:09.206159 (+ 1612us) server_negotiation.cc:407] Connection header received
0812 02:44:09.208083 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:09.208089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:09.208161 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:09.208287 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:09.209800 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:09.209806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:09.209818 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:09.209840 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:09.209880 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:09.209893 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:09.210049 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:09.210249 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 02:44:09.210327 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 02:44:09.211669 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:09.204724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:43359 (local address 127.17.196.194:39519)
0812 02:44:09.205993 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:09.206042 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:09.207791 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:09.208549 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:09.208558 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:09.208658 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:09.209412 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:44:09.209415 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:09.209456 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:44:09.209620 (+ 164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:09.209995 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:09.210013 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:09.210025 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 02:44:09.210355 (+ 330us) client_negotiation.cc:240] Negotiation successful
0812 02:44:09.210662 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":415,"threads_started":1}
I20260812 02:44:09.213335 29569 heartbeater.cc:347] Connected to a master server at 127.17.196.254:43359
I20260812 02:44:09.213793 29569 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:09.214594 29569 heartbeater.cc:511] Master 127.17.196.254:43359 requested a full tablet report, sending...
I20260812 02:44:09.216248 29214 ts_manager.cc:194] Registered new tserver with Master: 411418ef10ce495cbd309099b1e3d591 (127.17.196.194:35913)
W20260812 02:44:09.570226 29572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:09.570547 29572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:09.570721 29572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:09.579135 29572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:09.579250 29572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:09.579428 29572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:44:09.589108 29572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:43359
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:09.590514 29572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:09.592392 29572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:09.606649 29577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:09.606654 29578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:09.606496 29581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:09.608036 29572 server_base.cc:1034] running on GCE node
I20260812 02:44:09.608724 29572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:09.609825 29572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:09.611088 29572 hybrid_clock.cc:648] HybridClock initialized: now 1786502649611032 us; error 52 us; skew 500 ppm
I20260812 02:44:09.611534 29572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:44:09.614257 29572 webserver.cc:460] Webserver started at http://127.17.196.195:33885/ using document root <none> and password file <none>
I20260812 02:44:09.615047 29572 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:09.615196 29572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:09.615654 29572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:09.618819 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "7f6fec190b1a48658bbff0b61862e0ff"
format_stamp: "Formatted at 2026-08-12 02:44:09 on dist-test-slave-ss4n"
server_key: "80168459939c41eac987e6affefb111f"
server_key_iv: "dde8f48deba14f0c9d74d843bf0c0918"
server_key_version: "encryptionkey@0"
I20260812 02:44:09.619639 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "7f6fec190b1a48658bbff0b61862e0ff"
format_stamp: "Formatted at 2026-08-12 02:44:09 on dist-test-slave-ss4n"
server_key: "80168459939c41eac987e6affefb111f"
server_key_iv: "dde8f48deba14f0c9d74d843bf0c0918"
server_key_version: "encryptionkey@0"
I20260812 02:44:09.625411 29572 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:44:09.629323 29589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:09.630928 29572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:44:09.631124 29572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "7f6fec190b1a48658bbff0b61862e0ff"
format_stamp: "Formatted at 2026-08-12 02:44:09 on dist-test-slave-ss4n"
server_key: "80168459939c41eac987e6affefb111f"
server_key_iv: "dde8f48deba14f0c9d74d843bf0c0918"
server_key_version: "encryptionkey@0"
I20260812 02:44:09.631373 29572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:09.637857 29572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:09.638757 29572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:09.639050 29572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:09.640115 29572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:09.641661 29572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:09.641748 29572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:09.641912 29572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:09.641999 29572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:09.679311 29572 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:41843
I20260812 02:44:09.679481 29721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:41843 every 8 connection(s)
I20260812 02:44:09.680727 29572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:44:09.689383 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 29572
I20260812 02:44:09.689711 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:44:09.690268 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:09.683788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:46583 (local address 127.17.196.254:43359)
0812 02:44:09.683996 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:09.684001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:09.685558 (+ 1557us) server_negotiation.cc:407] Connection header received
0812 02:44:09.687570 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:09.687576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:09.687637 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:09.687741 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:09.689223 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:09.689229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:09.689235 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:09.689259 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:09.689302 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:09.689320 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:09.689469 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:09.689800 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 02:44:09.689880 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:44:09.690322 18195 external_mini_cluster.cc:1428] Setting key aa3cae73b9b66bc0e3adcc85d4d13b35
I20260812 02:44:09.691275 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:09.684315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:43359 (local address 127.17.196.195:46583)
0812 02:44:09.685333 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:09.685381 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:09.687183 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:09.687991 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:09.688000 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:09.688106 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:09.688856 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:44:09.688859 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:09.688900 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:44:09.688996 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:09.689435 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:09.689444 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:09.689450 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:44:09.689895 (+ 445us) client_negotiation.cc:240] Negotiation successful
0812 02:44:09.690146 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":324,"threads_started":1}
I20260812 02:44:09.692844 29722 heartbeater.cc:347] Connected to a master server at 127.17.196.254:43359
I20260812 02:44:09.693181 29722 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:09.693876 29722 heartbeater.cc:511] Master 127.17.196.254:43359 requested a full tablet report, sending...
I20260812 02:44:09.695477 29214 ts_manager.cc:194] Registered new tserver with Master: 7f6fec190b1a48658bbff0b61862e0ff (127.17.196.195:41843)
I20260812 02:44:09.706969 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 02:44:09.714165 29421 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:44:09.712238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35792 (local address 127.17.196.254:43359)
0812 02:44:09.712544 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:09.712553 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:09.712873 (+ 320us) server_negotiation.cc:407] Connection header received
0812 02:44:09.713011 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:09.713015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:09.713073 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:09.713167 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:09.713882 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35792: BlockingRecv error: recv got EOF from 127.0.0.1:35792 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
I20260812 02:44:09.714879 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 29279
I20260812 02:44:09.718941 29419 heartbeater.cc:503] Master 127.17.196.254:43359 was elected leader, sending a full tablet report...
I20260812 02:44:09.719079 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:09.715882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.17.196.193:45783)
0812 02:44:09.716518 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:09.716526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:09.716556 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:44:09.716627 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:09.716656 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:09.716885 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:09.717027 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:09.718180 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:09.718187 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:09.718199 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:09.718212 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:09.718259 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:09.718274 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:09.718417 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:09.718581 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 02:44:09.718673 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":281,"threads_started":1}
I20260812 02:44:09.720774 29413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:09.806242 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 29279
I20260812 02:44:09.828301 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 29423
I20260812 02:44:09.832439 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:09.829601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.17.196.194:35913)
0812 02:44:09.830151 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:09.830159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:09.830186 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:44:09.830250 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:09.830256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:09.830474 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:09.830604 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:09.831656 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:09.831663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:09.831675 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:09.831694 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:09.831738 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:09.831752 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:09.831862 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:09.831965 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 02:44:09.832118 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":237,"threads_started":1}
I20260812 02:44:09.833786 29564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:09.920651 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 29423
I20260812 02:44:09.943624 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 29572
I20260812 02:44:09.948758 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:09.944931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46882 (local address 127.17.196.195:41843)
0812 02:44:09.945965 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:09.945999 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:09.946042 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:44:09.946120 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:09.946128 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:09.946379 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:09.946667 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:09.947980 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:09.947988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:09.947999 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:44:09.948021 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:09.948068 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:44:09.948084 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:09.948246 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:09.948312 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 02:44:09.948412 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":444,"threads_started":1}
I20260812 02:44:09.950512 29716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:10.035869 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 29572
I20260812 02:44:10.057755 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 29176
I20260812 02:44:10.059332 29248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:10.167850 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 29176
2026-08-12T02:44:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2610 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:44:10 dist-test-slave-ss4n krb5kdc[29743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:44:10 dist-test-slave-ss4n krb5kdc[29743](info): set up 2 sockets
Aug 12 02:44:10 dist-test-slave-ss4n krb5kdc[29743](info): commencing operation
krb5kdc: starting...
W20260812 02:44:11.490288 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:44:11 dist-test-slave-ss4n krb5kdc[29743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:44:11.818312 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42821
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:42821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:44:12.185730 29759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:12.186057 29759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:12.186251 29759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:12.193694 29759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:44:12.193818 29759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:12.193948 29759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:12.194031 29759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:44:12.194100 29759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:44:12.205129 29759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:42821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:12.206638 29759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:12.208039 29759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:12.221005 29766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:12.221088 29767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:12.221102 29769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:12.222025 29759 server_base.cc:1034] running on GCE node
I20260812 02:44:12.222936 29759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:12.224491 29759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:12.225739 29759 hybrid_clock.cc:648] HybridClock initialized: now 1786502652225663 us; error 72 us; skew 500 ppm
I20260812 02:44:12.226145 29759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:44:12 dist-test-slave-ss4n krb5kdc[29743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502652, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:44:12.229981 29759 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:44:12.231765 29759 webserver.cc:460] Webserver started at http://127.17.196.254:35615/ using document root <none> and password file <none>
I20260812 02:44:12.232556 29759 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:12.232676 29759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:12.232967 29759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:12.236373 29759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "1a02e6e3154c46aead1bd10aa6903c20"
format_stamp: "Formatted at 2026-08-12 02:44:12 on dist-test-slave-ss4n"
server_key: "97aa6366f52c3ef2f35e19c069c6cca8"
server_key_iv: "a6e0a156741a9d42196422d390d6775d"
server_key_version: "encryptionkey@0"
I20260812 02:44:12.237253 29759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "1a02e6e3154c46aead1bd10aa6903c20"
format_stamp: "Formatted at 2026-08-12 02:44:12 on dist-test-slave-ss4n"
server_key: "97aa6366f52c3ef2f35e19c069c6cca8"
server_key_iv: "a6e0a156741a9d42196422d390d6775d"
server_key_version: "encryptionkey@0"
I20260812 02:44:12.243523 29759 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 02:44:12.247675 29776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:12.249238 29759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:44:12.249559 29759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "1a02e6e3154c46aead1bd10aa6903c20"
format_stamp: "Formatted at 2026-08-12 02:44:12 on dist-test-slave-ss4n"
server_key: "97aa6366f52c3ef2f35e19c069c6cca8"
server_key_iv: "a6e0a156741a9d42196422d390d6775d"
server_key_version: "encryptionkey@0"
I20260812 02:44:12.249795 29759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:12.280952 29759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:12.286077 29759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:12.286414 29759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:12.307372 29759 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:42821
I20260812 02:44:12.307533 29833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:42821 every 8 connection(s)
I20260812 02:44:12.308838 29759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:44:12.310938 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 29759
I20260812 02:44:12.311265 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:44:12.312243 18195 external_mini_cluster.cc:1428] Setting key bd80494cdf0614d8d97433ea43ece682
I20260812 02:44:12.314378 29834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:44:12 dist-test-slave-ss4n krb5kdc[29743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:44:12.330049 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20: Bootstrap starting.
I20260812 02:44:12.333860 29834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20: Neither blocks nor log segments found. Creating new log.
I20260812 02:44:12.335057 29834 log.cc:824] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20: Log is configured to *not* fsync() on all Append() calls
I20260812 02:44:12.338418 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20: No bootstrap required, opened a new log
I20260812 02:44:12.338611 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:12.314670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.17.196.254:42821)
0812 02:44:12.315505 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:12.315518 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:12.315562 (+ 44us) server_negotiation.cc:407] Connection header received
0812 02:44:12.317269 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:12.317294 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:12.317793 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:12.318161 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:12.320416 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:12.322117 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:12.324091 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:12.324868 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:12.325049 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:12.330532 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:12.330541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:12.330561 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:12.330610 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:12.334063 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:12.334899 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:12.334912 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:12.334920 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:12.335129 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:12.335552 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:12.335559 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:12.335567 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:12.336155 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:12.336516 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:12.336993 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 02:44:12.337471 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":353,"threads_started":1}
I20260812 02:44:12.347263 29834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02e6e3154c46aead1bd10aa6903c20" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42821 } }
I20260812 02:44:12.347730 29834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:44:12.347846 29834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a02e6e3154c46aead1bd10aa6903c20, State: Initialized, Role: FOLLOWER
I20260812 02:44:12.348567 29834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02e6e3154c46aead1bd10aa6903c20" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42821 } }
I20260812 02:44:12.348835 29834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:44:12.349004 29834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:44:12.349200 29834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:44:12.351897 29834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02e6e3154c46aead1bd10aa6903c20" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42821 } }
I20260812 02:44:12.352555 29834 leader_election.cc:302] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a02e6e3154c46aead1bd10aa6903c20; no voters:
I20260812 02:44:12.353272 29834 leader_election.cc:288] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:44:12.353515 29840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:44:12.354415 29840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [term 1 LEADER]: Becoming Leader. State: Replica: 1a02e6e3154c46aead1bd10aa6903c20, State: Running, Role: LEADER
I20260812 02:44:12.355156 29840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02e6e3154c46aead1bd10aa6903c20" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42821 } }
I20260812 02:44:12.355683 29834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:44:12.358539 29842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a02e6e3154c46aead1bd10aa6903c20. Latest consensus state: current_term: 1 leader_uuid: "1a02e6e3154c46aead1bd10aa6903c20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02e6e3154c46aead1bd10aa6903c20" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42821 } } }
I20260812 02:44:12.358692 29841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a02e6e3154c46aead1bd10aa6903c20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02e6e3154c46aead1bd10aa6903c20" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42821 } } }
I20260812 02:44:12.359030 29841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [sys.catalog]: This master's current role is: LEADER
I20260812 02:44:12.359030 29842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20 [sys.catalog]: This master's current role is: LEADER
I20260812 02:44:12.360175 29849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:44:12.366914 29849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:44:12.377446 29849 catalog_manager.cc:1347] Generated new cluster ID: 5bed76cb27d649099c8b2e3a93d2a464
I20260812 02:44:12.377632 29849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:44:12.421450 29849 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:44:12.422936 29849 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:44:12.430361 29849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20: Generated new TSK 0
I20260812 02:44:12.431380 29849 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:44:12.503371 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42821
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:44:12.890336 29865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:12.890628 29865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:12.890724 29865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:12.898633 29865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:12.898837 29865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:12.899036 29865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:44:12.909288 29865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:12.910794 29865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:12.912196 29865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:12.926998 29870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:12.926923 29871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:12.927037 29873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:12.928269 29865 server_base.cc:1034] running on GCE node
I20260812 02:44:12.929093 29865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:12.930330 29865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:12.931658 29865 hybrid_clock.cc:648] HybridClock initialized: now 1786502652931603 us; error 44 us; skew 500 ppm
I20260812 02:44:12.932056 29865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:44:12 dist-test-slave-ss4n krb5kdc[29743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502652, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:44:12.935940 29865 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:44:12.937749 29865 webserver.cc:460] Webserver started at http://127.17.196.193:38693/ using document root <none> and password file <none>
I20260812 02:44:12.938582 29865 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:12.938756 29865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:12.939113 29865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:12.942647 29865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "11f1ec392d9a45f29a629fadc33fac79"
format_stamp: "Formatted at 2026-08-12 02:44:12 on dist-test-slave-ss4n"
server_key: "db5a17820d47765c4b95752484ada5ba"
server_key_iv: "8904154093e89c236baa8ff4ac15680d"
server_key_version: "encryptionkey@0"
I20260812 02:44:12.943584 29865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "11f1ec392d9a45f29a629fadc33fac79"
format_stamp: "Formatted at 2026-08-12 02:44:12 on dist-test-slave-ss4n"
server_key: "db5a17820d47765c4b95752484ada5ba"
server_key_iv: "8904154093e89c236baa8ff4ac15680d"
server_key_version: "encryptionkey@0"
I20260812 02:44:12.949754 29865 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260812 02:44:12.954325 29880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:12.956053 29865 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:44:12.956301 29865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "11f1ec392d9a45f29a629fadc33fac79"
format_stamp: "Formatted at 2026-08-12 02:44:12 on dist-test-slave-ss4n"
server_key: "db5a17820d47765c4b95752484ada5ba"
server_key_iv: "8904154093e89c236baa8ff4ac15680d"
server_key_version: "encryptionkey@0"
I20260812 02:44:12.956619 29865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:12.980876 29865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:12.985992 29865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:12.986383 29865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:12.987622 29865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:12.989296 29865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:12.989419 29865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:12.989602 29865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:12.989681 29865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:13.027482 29865 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:43779
I20260812 02:44:13.027612 30007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:43779 every 8 connection(s)
I20260812 02:44:13.028901 29865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:44:13.038061 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 29865
I20260812 02:44:13.038367 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:44:13.039389 18195 external_mini_cluster.cc:1428] Setting key f1703da8276d5c7661bf5f0eae878f90
Aug 12 02:44:13 dist-test-slave-ss4n krb5kdc[29743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502652, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:44:13.053972 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:13.031957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:37227 (local address 127.17.196.254:42821)
0812 02:44:13.032503 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:13.032508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:13.034069 (+ 1561us) server_negotiation.cc:407] Connection header received
0812 02:44:13.036195 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:13.036204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:13.036319 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:13.036473 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:13.038787 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:13.039643 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:13.042022 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:13.042503 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:13.042678 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:13.046889 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:13.046896 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:13.046899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:13.046942 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:13.050097 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:13.051076 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:13.051082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:13.051086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:13.051235 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:13.051780 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:13.051784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:13.051787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:13.052093 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:13.052282 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:13.053184 (+ 902us) server_negotiation.cc:299] Negotiation successful
0812 02:44:13.053457 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":253,"threads_started":1}
I20260812 02:44:13.055357 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:13.032604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42821 (local address 127.17.196.193:37227)
0812 02:44:13.033859 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:13.033917 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:13.035755 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:13.036836 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:13.036851 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:13.037659 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:13.038531 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:13.038570 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:13.039873 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:13.039879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:13.041835 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:13.041851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:13.042811 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:13.042819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:13.043101 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:13.043924 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:13.043951 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:13.046584 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:13.050325 (+ 3741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:44:13.050338 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:44:13.050368 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:44:13.050904 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:44:13.051378 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:44:13.051385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:44:13.051389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:44:13.051628 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:44:13.052256 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:13.052267 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:13.052692 (+ 425us) client_negotiation.cc:769] Sending connection context
0812 02:44:13.053068 (+ 376us) client_negotiation.cc:240] Negotiation successful
0812 02:44:13.053781 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"thread_start_us":555,"threads_started":1}
I20260812 02:44:13.057627 30008 heartbeater.cc:347] Connected to a master server at 127.17.196.254:42821
I20260812 02:44:13.058017 30008 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:13.059026 30008 heartbeater.cc:511] Master 127.17.196.254:42821 requested a full tablet report, sending...
I20260812 02:44:13.061950 29793 ts_manager.cc:194] Registered new tserver with Master: 11f1ec392d9a45f29a629fadc33fac79 (127.17.196.193:43779)
I20260812 02:44:13.064302 29793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:37227
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:44:13.119138 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42821
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:44:13.501724 30015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:13.502094 30015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:13.502223 30015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:13.510787 30015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:13.510926 30015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:13.511044 30015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:44:13.521749 30015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:13.523087 30015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:13.524411 30015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:13.538447 30021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:13.538533 30022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:13.538491 30024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:13.539942 30015 server_base.cc:1034] running on GCE node
I20260812 02:44:13.540678 30015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:13.541821 30015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:13.543077 30015 hybrid_clock.cc:648] HybridClock initialized: now 1786502653543015 us; error 55 us; skew 500 ppm
I20260812 02:44:13.543418 30015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:44:13 dist-test-slave-ss4n krb5kdc[29743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502653, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:44:13.546944 30015 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:44:13.548702 30015 webserver.cc:460] Webserver started at http://127.17.196.194:34595/ using document root <none> and password file <none>
I20260812 02:44:13.549562 30015 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:13.549719 30015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:13.550100 30015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:13.553264 30015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "8127bd25e4cb41c2be36eb01435c60c4"
format_stamp: "Formatted at 2026-08-12 02:44:13 on dist-test-slave-ss4n"
server_key: "7db17a9ad7d42777acbcccf76ac40b15"
server_key_iv: "647fb6cafd80945b3d7ee868d732c797"
server_key_version: "encryptionkey@0"
I20260812 02:44:13.554203 30015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "8127bd25e4cb41c2be36eb01435c60c4"
format_stamp: "Formatted at 2026-08-12 02:44:13 on dist-test-slave-ss4n"
server_key: "7db17a9ad7d42777acbcccf76ac40b15"
server_key_iv: "647fb6cafd80945b3d7ee868d732c797"
server_key_version: "encryptionkey@0"
I20260812 02:44:13.560070 30015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:44:13.564481 30035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:13.566285 30015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:44:13.566511 30015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "8127bd25e4cb41c2be36eb01435c60c4"
format_stamp: "Formatted at 2026-08-12 02:44:13 on dist-test-slave-ss4n"
server_key: "7db17a9ad7d42777acbcccf76ac40b15"
server_key_iv: "647fb6cafd80945b3d7ee868d732c797"
server_key_version: "encryptionkey@0"
I20260812 02:44:13.566735 30015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:13.582435 30015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:13.587743 30015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:13.588100 30015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:13.589198 30015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:13.590803 30015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:13.590888 30015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:13.591078 30015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:13.591161 30015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:13.628291 30015 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:41817
I20260812 02:44:13.628340 30161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:41817 every 8 connection(s)
I20260812 02:44:13.629868 30015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:44:13.636257 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 30015
I20260812 02:44:13.636485 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:44:13.636979 18195 external_mini_cluster.cc:1428] Setting key 579b50b0fdfe0d5d8696e6dd40ee213f
Aug 12 02:44:13 dist-test-slave-ss4n krb5kdc[29743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502653, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:44:13.658200 30164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:13.633014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:38691 (local address 127.17.196.254:42821)
0812 02:44:13.633604 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:13.633609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:13.634671 (+ 1062us) server_negotiation.cc:407] Connection header received
0812 02:44:13.636653 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:13.636658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:13.636754 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:13.636882 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:13.638614 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:13.639410 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:13.641038 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:13.641550 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:13.641698 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:13.650393 (+ 8695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:13.650404 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:13.650410 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:13.650469 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:13.654221 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:13.655297 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:13.655306 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:13.655312 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:13.655489 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:13.655951 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:13.655958 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:13.655964 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:13.656398 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:13.656568 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:13.657369 (+ 801us) server_negotiation.cc:299] Negotiation successful
0812 02:44:13.657692 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":250,"threads_started":1}
I20260812 02:44:13.659054 30165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:13.633710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42821 (local address 127.17.196.194:38691)
0812 02:44:13.634466 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:13.634526 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:13.636319 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:13.637183 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:13.637192 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:13.637809 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:13.638431 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:13.638450 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:13.639584 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:13.639589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:13.640861 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:13.640875 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:13.641792 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:13.641800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:13.642062 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:13.647253 (+ 5191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:13.647283 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:13.650060 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:13.654449 (+ 4389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:44:13.654459 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:44:13.654474 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:44:13.655135 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:44:13.655612 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:44:13.655616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:44:13.655618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:44:13.655818 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:44:13.656579 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:13.656586 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:13.656945 (+ 359us) client_negotiation.cc:769] Sending connection context
0812 02:44:13.657261 (+ 316us) client_negotiation.cc:240] Negotiation successful
0812 02:44:13.657805 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":302,"threads_started":1}
I20260812 02:44:13.660877 30162 heartbeater.cc:347] Connected to a master server at 127.17.196.254:42821
I20260812 02:44:13.661278 30162 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:13.662367 30162 heartbeater.cc:511] Master 127.17.196.254:42821 requested a full tablet report, sending...
I20260812 02:44:13.664134 29793 ts_manager.cc:194] Registered new tserver with Master: 8127bd25e4cb41c2be36eb01435c60c4 (127.17.196.194:41817)
I20260812 02:44:13.665019 29793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:38691
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:44:13.710690 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42821
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:44:14.068357 30008 heartbeater.cc:503] Master 127.17.196.254:42821 was elected leader, sending a full tablet report...
W20260812 02:44:14.078153 30170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:44:14.078526 30170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:44:14.078692 30170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:44:14.086377 30170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:44:14.086560 30170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:44:14.086759 30170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:44:14.097043 30170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:42821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:44:14.098572 30170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:44:14.099918 30170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:44:14.114578 30181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:14.114718 30176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:44:14.114981 30175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:44:14.115890 30170 server_base.cc:1034] running on GCE node
I20260812 02:44:14.116647 30170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:44:14.117758 30170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:44:14.119025 30170 hybrid_clock.cc:648] HybridClock initialized: now 1786502654118983 us; error 44 us; skew 500 ppm
I20260812 02:44:14.119453 30170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:44:14 dist-test-slave-ss4n krb5kdc[29743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502654, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:44:14.123397 30170 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:44:14.125101 30170 webserver.cc:460] Webserver started at http://127.17.196.195:46047/ using document root <none> and password file <none>
I20260812 02:44:14.125962 30170 fs_manager.cc:359] Metadata directory not provided
I20260812 02:44:14.126118 30170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:44:14.126446 30170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:44:14.129591 30170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "96ec6f967f2d40a89df547282ce4133c"
format_stamp: "Formatted at 2026-08-12 02:44:14 on dist-test-slave-ss4n"
server_key: "a616390d1b369d9571878b84356ad4a5"
server_key_iv: "c730909fed2be34d83905052df0b480a"
server_key_version: "encryptionkey@0"
I20260812 02:44:14.130486 30170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "96ec6f967f2d40a89df547282ce4133c"
format_stamp: "Formatted at 2026-08-12 02:44:14 on dist-test-slave-ss4n"
server_key: "a616390d1b369d9571878b84356ad4a5"
server_key_iv: "c730909fed2be34d83905052df0b480a"
server_key_version: "encryptionkey@0"
I20260812 02:44:14.136149 30170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:44:14.140113 30189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:14.141703 30170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:44:14.141901 30170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "96ec6f967f2d40a89df547282ce4133c"
format_stamp: "Formatted at 2026-08-12 02:44:14 on dist-test-slave-ss4n"
server_key: "a616390d1b369d9571878b84356ad4a5"
server_key_iv: "c730909fed2be34d83905052df0b480a"
server_key_version: "encryptionkey@0"
I20260812 02:44:14.142128 30170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:44:14.165416 30170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:44:14.170353 30170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:44:14.170740 30170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:44:14.171878 30170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:44:14.173372 30170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:44:14.173483 30170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:14.173664 30170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:44:14.173739 30170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:44:14.209810 30170 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:40623
I20260812 02:44:14.209909 30316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:40623 every 8 connection(s)
I20260812 02:44:14.211215 30170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:44:14.213078 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 30170
I20260812 02:44:14.213310 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:44:14.213908 18195 external_mini_cluster.cc:1428] Setting key 8c3c1327311cb7bf5bada1ae1f40fe8f
Aug 12 02:44:14 dist-test-slave-ss4n krb5kdc[29743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502654, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:44:14.234546 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.214351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:54853 (local address 127.17.196.254:42821)
0812 02:44:14.214902 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.214908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.216327 (+ 1419us) server_negotiation.cc:407] Connection header received
0812 02:44:14.218731 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.218737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.218873 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.219012 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:14.220695 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.221633 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.223425 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.223905 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.224070 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.228108 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:14.228118 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:14.228125 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:14.228191 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:14.231436 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.232225 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.232228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.232231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.232355 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.232786 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.232789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.232791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.233065 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:14.233184 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.234002 (+ 818us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.234190 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":295,"threads_started":1}
I20260812 02:44:14.235601 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.215016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:42821 (local address 127.17.196.195:54853)
0812 02:44:14.216124 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:14.216177 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:14.218435 (+ 2258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:14.219296 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:14.219305 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:14.219870 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:44:14.220496 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.220520 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.221808 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.221822 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.223253 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.223268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.224191 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.224202 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.224494 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.225253 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:44:14.225279 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:44:14.227841 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:44:14.231586 (+ 3745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:44:14.231594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:44:14.231610 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:44:14.232114 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:44:14.232453 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:44:14.232456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:44:14.232459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:44:14.232685 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:44:14.233184 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:44:14.233190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:44:14.233642 (+ 452us) client_negotiation.cc:769] Sending connection context
0812 02:44:14.233916 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 02:44:14.234364 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":509,"threads_started":1}
I20260812 02:44:14.237094 30317 heartbeater.cc:347] Connected to a master server at 127.17.196.254:42821
I20260812 02:44:14.237437 30317 heartbeater.cc:464] Registering TS with master...
I20260812 02:44:14.238294 30317 heartbeater.cc:511] Master 127.17.196.254:42821 requested a full tablet report, sending...
I20260812 02:44:14.239969 29793 ts_manager.cc:194] Registered new tserver with Master: 96ec6f967f2d40a89df547282ce4133c (127.17.196.195:40623)
I20260812 02:44:14.241094 29793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:54853
I20260812 02:44:14.243263 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:44:14.263077 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.248696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.17.196.254:42821)
0812 02:44:14.248872 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.248877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.249454 (+ 577us) server_negotiation.cc:407] Connection header received
0812 02:44:14.249680 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.249684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.249778 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.249867 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:14.251403 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.252135 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.253565 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.254082 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.254196 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.256934 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:14.256938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:14.256941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:14.256973 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:14.259706 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.260336 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.260339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.260342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.260465 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.261020 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.261025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.261027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.261651 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:14.261882 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.262409 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.262724 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:44:14.267927 29792 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:44:14.275429 29792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:44:14.315790 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.289555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:41817 (local address 127.0.0.1:45740)
0812 02:44:14.302364 (+ 12809us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:14.302401 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:14.302633 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:14.303871 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:14.303883 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:14.303925 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:44:14.304482 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.304499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.308321 (+ 3822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.308328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.312192 (+ 3864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.312211 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.314576 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.314583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.314860 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.314932 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 02:44:14.315073 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 02:44:14.315301 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12505,"mutex_wait_us":28,"thread_start_us":15457,"threads_started":1}
I20260812 02:44:14.316637 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.289815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.17.196.194:41817)
0812 02:44:14.302723 (+ 12908us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.302734 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.302769 (+ 35us) server_negotiation.cc:407] Connection header received
0812 02:44:14.302901 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.302910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.303232 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.303507 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:44:14.304736 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.306627 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.312437 (+ 5810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.313989 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.314194 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.314409 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.316069 (+ 1660us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.316276 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12710,"thread_start_us":13817,"threads_started":1}
I20260812 02:44:14.316670 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.291500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.17.196.195:40623)
0812 02:44:14.303472 (+ 11972us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.303485 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.304886 (+ 1401us) server_negotiation.cc:407] Connection header received
0812 02:44:14.305362 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.305371 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.305653 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.305823 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:44:14.306913 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.308742 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.312592 (+ 3850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.314340 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.314553 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.314746 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.316065 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.316273 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11688,"thread_start_us":11886,"threads_started":1}
I20260812 02:44:14.317292 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.289539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54890 (local address 127.17.196.193:43779)
0812 02:44:14.304343 (+ 14804us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.304354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.304389 (+ 35us) server_negotiation.cc:407] Connection header received
0812 02:44:14.304503 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.304511 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.304768 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.305006 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:44:14.305723 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.307221 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.310445 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.315188 (+ 4743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.316449 (+ 1261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.316639 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.316749 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.316930 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14617,"thread_start_us":14429,"threads_started":1}
I20260812 02:44:14.315912 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.290762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40623 (local address 127.0.0.1:54056)
0812 02:44:14.304727 (+ 13965us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:14.304749 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:14.304907 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:14.306217 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:14.306223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:14.306251 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:44:14.306637 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.306650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.308929 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.308934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.312334 (+ 3400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.312352 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.314918 (+ 2566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.314925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.315123 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.315160 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:44:14.315252 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 02:44:14.315412 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13805,"mutex_wait_us":66,"thread_start_us":11051,"threads_started":1}
I20260812 02:44:14.316273 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.289144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43779 (local address 127.0.0.1:54890)
0812 02:44:14.290217 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:14.290257 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:14.290452 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:14.304994 (+ 14542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:14.304999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:14.305030 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:44:14.305456 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.305470 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.307400 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.307407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.310257 (+ 2850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.310271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.315460 (+ 5189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.315467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.315645 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.315678 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:44:14.315764 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 02:44:14.315936 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":812,"threads_started":1}
I20260812 02:44:14.321623 29934 tablet_service.cc:1467] Processing CreateTablet for tablet 59eebed06f1f4f0894584de4c1a6dd6f (DEFAULT_TABLE table=test-table [id=5b65400307d249cbb7fe58c172d4d460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:44:14.321623 30241 tablet_service.cc:1467] Processing CreateTablet for tablet 59eebed06f1f4f0894584de4c1a6dd6f (DEFAULT_TABLE table=test-table [id=5b65400307d249cbb7fe58c172d4d460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:44:14.321573 30088 tablet_service.cc:1467] Processing CreateTablet for tablet 59eebed06f1f4f0894584de4c1a6dd6f (DEFAULT_TABLE table=test-table [id=5b65400307d249cbb7fe58c172d4d460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:44:14.323230 30241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59eebed06f1f4f0894584de4c1a6dd6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:44:14.323232 30088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59eebed06f1f4f0894584de4c1a6dd6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:44:14.323249 29934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59eebed06f1f4f0894584de4c1a6dd6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:44:14.338526 30340 tablet_bootstrap.cc:492] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Bootstrap starting.
I20260812 02:44:14.340817 30341 tablet_bootstrap.cc:492] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: Bootstrap starting.
I20260812 02:44:14.343364 30340 tablet_bootstrap.cc:654] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Neither blocks nor log segments found. Creating new log.
I20260812 02:44:14.345079 30341 tablet_bootstrap.cc:654] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: Neither blocks nor log segments found. Creating new log.
I20260812 02:44:14.346010 30340 log.cc:824] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:44:14.346717 30341 log.cc:824] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: Log is configured to *not* fsync() on all Append() calls
I20260812 02:44:14.348582 30342 tablet_bootstrap.cc:492] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Bootstrap starting.
I20260812 02:44:14.350373 30340 tablet_bootstrap.cc:492] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: No bootstrap required, opened a new log
I20260812 02:44:14.350591 30341 tablet_bootstrap.cc:492] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: No bootstrap required, opened a new log
I20260812 02:44:14.350720 30340 ts_tablet_manager.cc:1397] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 02:44:14.350901 30341 ts_tablet_manager.cc:1397] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.001s
I20260812 02:44:14.352864 30342 tablet_bootstrap.cc:654] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Neither blocks nor log segments found. Creating new log.
I20260812 02:44:14.354651 30342 log.cc:824] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Log is configured to *not* fsync() on all Append() calls
I20260812 02:44:14.358620 30342 tablet_bootstrap.cc:492] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: No bootstrap required, opened a new log
I20260812 02:44:14.358968 30342 ts_tablet_manager.cc:1397] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260812 02:44:14.359661 30341 raft_consensus.cc:348] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.360078 30341 raft_consensus.cc:374] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:44:14.360213 30341 raft_consensus.cc:729] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f1ec392d9a45f29a629fadc33fac79, State: Initialized, Role: FOLLOWER
I20260812 02:44:14.361074 30341 consensus_queue.cc:260] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.363545 30341 ts_tablet_manager.cc:1428] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
I20260812 02:44:14.363435 30340 raft_consensus.cc:348] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.364096 30340 raft_consensus.cc:374] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:44:14.364289 30340 raft_consensus.cc:729] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8127bd25e4cb41c2be36eb01435c60c4, State: Initialized, Role: FOLLOWER
I20260812 02:44:14.367900 30340 consensus_queue.cc:260] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.370354 30162 heartbeater.cc:503] Master 127.17.196.254:42821 was elected leader, sending a full tablet report...
I20260812 02:44:14.371238 30340 ts_tablet_manager.cc:1428] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Time spent starting tablet: real 0.020s user 0.010s sys 0.008s
I20260812 02:44:14.371840 30342 raft_consensus.cc:348] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.372287 30342 raft_consensus.cc:374] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:44:14.372407 30342 raft_consensus.cc:729] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ec6f967f2d40a89df547282ce4133c, State: Initialized, Role: FOLLOWER
I20260812 02:44:14.373052 30342 consensus_queue.cc:260] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.374361 30317 heartbeater.cc:503] Master 127.17.196.254:42821 was elected leader, sending a full tablet report...
I20260812 02:44:14.375185 30342 ts_tablet_manager.cc:1428] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
W20260812 02:44:14.382019 30163 tablet.cc:2369] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:44:14.401405 30346 raft_consensus.cc:482] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:44:14.401823 30346 raft_consensus.cc:504] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.403748 30346 leader_election.cc:288] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8127bd25e4cb41c2be36eb01435c60c4 (127.17.196.194:41817), 96ec6f967f2d40a89df547282ce4133c (127.17.196.195:40623)
I20260812 02:44:14.410696 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.404257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:40297 (local address 127.17.196.195:40623)
0812 02:44:14.404491 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.404495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.405159 (+ 664us) server_negotiation.cc:407] Connection header received
0812 02:44:14.405405 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.405411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.405536 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.405653 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:44:14.406214 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.406979 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.408805 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.409729 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.409921 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.410015 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.410290 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.410418 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:44:14.410776 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.404139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40623 (local address 127.17.196.193:40297)
0812 02:44:14.405029 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:14.405047 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:14.405247 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:14.405672 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:14.405677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:14.405716 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:44:14.406065 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.406074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.407124 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.407131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.408634 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.408648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.409938 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.409944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.410115 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.410140 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:44:14.410263 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 02:44:14.410400 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":537,"threads_started":1}
I20260812 02:44:14.410959 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.404142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:32907 (local address 127.17.196.194:41817)
0812 02:44:14.404392 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.404400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.405689 (+ 1289us) server_negotiation.cc:407] Connection header received
0812 02:44:14.405961 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.405969 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.406035 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.406137 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:44:14.406665 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.407422 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.409162 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.410015 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.410195 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.410274 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.410566 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.410699 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:44:14.411335 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.404650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:41817 (local address 127.17.196.193:32907)
0812 02:44:14.405571 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0812 02:44:14.405587 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:44:14.405802 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:44:14.406155 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:44:14.406159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:44:14.406183 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:44:14.406493 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.406501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.407583 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.407589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.408988 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:44:14.409001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.410243 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.410249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:44:14.410399 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.410421 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 02:44:14.410529 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 02:44:14.410658 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"spinlock_wait_cycles":26496,"thread_start_us":494,"threads_started":1}
I20260812 02:44:14.411978 30263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59eebed06f1f4f0894584de4c1a6dd6f" candidate_uuid: "11f1ec392d9a45f29a629fadc33fac79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ec6f967f2d40a89df547282ce4133c" is_pre_election: true
I20260812 02:44:14.412313 30112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59eebed06f1f4f0894584de4c1a6dd6f" candidate_uuid: "11f1ec392d9a45f29a629fadc33fac79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8127bd25e4cb41c2be36eb01435c60c4" is_pre_election: true
I20260812 02:44:14.412505 30263 raft_consensus.cc:2454] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f1ec392d9a45f29a629fadc33fac79 in term 0.
I20260812 02:44:14.412845 30112 raft_consensus.cc:2454] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f1ec392d9a45f29a629fadc33fac79 in term 0.
I20260812 02:44:14.413296 29883 leader_election.cc:302] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f1ec392d9a45f29a629fadc33fac79, 96ec6f967f2d40a89df547282ce4133c; no voters:
I20260812 02:44:14.413782 30346 raft_consensus.cc:2781] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:44:14.413949 30346 raft_consensus.cc:482] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:44:14.414057 30346 raft_consensus.cc:3037] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:44:14.416359 30346 raft_consensus.cc:504] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.417081 30346 leader_election.cc:288] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [CANDIDATE]: Term 1 election: Requested vote from peers 8127bd25e4cb41c2be36eb01435c60c4 (127.17.196.194:41817), 96ec6f967f2d40a89df547282ce4133c (127.17.196.195:40623)
I20260812 02:44:14.417641 30263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59eebed06f1f4f0894584de4c1a6dd6f" candidate_uuid: "11f1ec392d9a45f29a629fadc33fac79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ec6f967f2d40a89df547282ce4133c"
I20260812 02:44:14.417666 30112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59eebed06f1f4f0894584de4c1a6dd6f" candidate_uuid: "11f1ec392d9a45f29a629fadc33fac79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8127bd25e4cb41c2be36eb01435c60c4"
I20260812 02:44:14.417850 30263 raft_consensus.cc:3037] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:44:14.417924 30112 raft_consensus.cc:3037] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:44:14.420174 30263 raft_consensus.cc:2454] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f1ec392d9a45f29a629fadc33fac79 in term 1.
I20260812 02:44:14.420542 30112 raft_consensus.cc:2454] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f1ec392d9a45f29a629fadc33fac79 in term 1.
I20260812 02:44:14.420606 29883 leader_election.cc:302] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f1ec392d9a45f29a629fadc33fac79, 96ec6f967f2d40a89df547282ce4133c; no voters:
I20260812 02:44:14.420893 30346 raft_consensus.cc:2781] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:44:14.421375 30346 raft_consensus.cc:686] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 1 LEADER]: Becoming Leader. State: Replica: 11f1ec392d9a45f29a629fadc33fac79, State: Running, Role: LEADER
I20260812 02:44:14.421893 30346 consensus_queue.cc:237] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } }
I20260812 02:44:14.426959 29792 catalog_manager.cc:5495] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f1ec392d9a45f29a629fadc33fac79 (127.17.196.193). New cstate: current_term: 1 leader_uuid: "11f1ec392d9a45f29a629fadc33fac79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f1ec392d9a45f29a629fadc33fac79" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 } health_report { overall_health: UNKNOWN } } }
I20260812 02:44:14.452315 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.446608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.17.196.193:43779)
0812 02:44:14.446852 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.446860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.447041 (+ 181us) server_negotiation.cc:407] Connection header received
0812 02:44:14.447266 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.447270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.447350 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.447482 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:44:14.447825 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.448569 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.449952 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.450365 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.450474 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.451099 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:44:14.451649 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:44:14.451764 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.451901 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.451975 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:44:14.462181 30112 raft_consensus.cc:1261] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 1 FOLLOWER]: Refusing update from remote peer 11f1ec392d9a45f29a629fadc33fac79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:44:14.462189 30263 raft_consensus.cc:1261] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 1 FOLLOWER]: Refusing update from remote peer 11f1ec392d9a45f29a629fadc33fac79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 02:44:14.462985 30318 tablet.cc:2369] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:44:14.463460 30351 consensus_queue.cc:1035] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ec6f967f2d40a89df547282ce4133c" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:44:14.463939 30346 consensus_queue.cc:1035] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8127bd25e4cb41c2be36eb01435c60c4" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 41817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:44:14.476292 30355 mvcc.cc:205] Tried to move back new op lower bound from 7317514872664313856 to 7317514872515919872. Current Snapshot: MvccSnapshot[applied={T|T < 7317514872664313856}]
I20260812 02:44:14.493821 29792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:44:14.494210 29792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:44:14.498210 29792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a02e6e3154c46aead1bd10aa6903c20: Sending DeleteTablet for 3 replicas of tablet 59eebed06f1f4f0894584de4c1a6dd6f
I20260812 02:44:14.499614 30088 tablet_service.cc:1514] Processing DeleteTablet for tablet 59eebed06f1f4f0894584de4c1a6dd6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:44:14 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:45740
I20260812 02:44:14.500082 29934 tablet_service.cc:1514] Processing DeleteTablet for tablet 59eebed06f1f4f0894584de4c1a6dd6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:44:14 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:54890
I20260812 02:44:14.500846 30241 tablet_service.cc:1514] Processing DeleteTablet for tablet 59eebed06f1f4f0894584de4c1a6dd6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:44:14 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:54056
I20260812 02:44:14.501210 30368 tablet_replica.cc:330] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: stopping tablet replica
I20260812 02:44:14.501863 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 29865
I20260812 02:44:14.501806 30368 raft_consensus.cc:2229] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:44:14.502656 30368 raft_consensus.cc:2258] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:44:14.505375 30368 ts_tablet_manager.cc:1905] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:44:14.509820 30369 tablet_replica.cc:330] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: stopping tablet replica
I20260812 02:44:14.510325 30369 raft_consensus.cc:2229] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:44:14.510870 30369 raft_consensus.cc:2258] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:44:14.515583 30369 ts_tablet_manager.cc:1905] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:44:14.515857 30368 ts_tablet_manager.cc:1918] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:44:14.516606 30367 tablet_replica.cc:330] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: stopping tablet replica
I20260812 02:44:14.517231 30367 raft_consensus.cc:2229] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:44:14.518242 30367 raft_consensus.cc:2258] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:44:14.520219 30368 log.cc:1196] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/59eebed06f1f4f0894584de4c1a6dd6f
I20260812 02:44:14.520634 30367 ts_tablet_manager.cc:1905] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:44:14.520852 30368 ts_tablet_manager.cc:1939] T 59eebed06f1f4f0894584de4c1a6dd6f P 11f1ec392d9a45f29a629fadc33fac79: Deleting consensus metadata
Aug 12 02:44:14 dist-test-slave-ss4n krb5kdc[29743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:44:14.523811 30369 ts_tablet_manager.cc:1918] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:44:14.524111 30369 log.cc:1196] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/59eebed06f1f4f0894584de4c1a6dd6f
I20260812 02:44:14.524962 30369 ts_tablet_manager.cc:1939] T 59eebed06f1f4f0894584de4c1a6dd6f P 96ec6f967f2d40a89df547282ce4133c: Deleting consensus metadata
I20260812 02:44:14.525359 29779 catalog_manager.cc:4841] TS 11f1ec392d9a45f29a629fadc33fac79 (127.17.196.193:43779): tablet 59eebed06f1f4f0894584de4c1a6dd6f (table test-table [id=5b65400307d249cbb7fe58c172d4d460]) successfully deleted
I20260812 02:44:14.527281 29779 catalog_manager.cc:4841] TS 96ec6f967f2d40a89df547282ce4133c (127.17.196.195:40623): tablet 59eebed06f1f4f0894584de4c1a6dd6f (table test-table [id=5b65400307d249cbb7fe58c172d4d460]) successfully deleted
I20260812 02:44:14.528970 30367 ts_tablet_manager.cc:1918] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:44:14.529153 30367 log.cc:1196] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/59eebed06f1f4f0894584de4c1a6dd6f
I20260812 02:44:14.529812 30367 ts_tablet_manager.cc:1939] T 59eebed06f1f4f0894584de4c1a6dd6f P 8127bd25e4cb41c2be36eb01435c60c4: Deleting consensus metadata
I20260812 02:44:14.531813 29780 catalog_manager.cc:4841] TS 8127bd25e4cb41c2be36eb01435c60c4 (127.17.196.194:41817): tablet 59eebed06f1f4f0894584de4c1a6dd6f (table test-table [id=5b65400307d249cbb7fe58c172d4d460]) successfully deleted
I20260812 02:44:14.532388 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.504462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.17.196.193:43779)
0812 02:44:14.504662 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.504669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.510029 (+ 5360us) server_negotiation.cc:407] Connection header received
0812 02:44:14.510280 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.510286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.510388 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.510500 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:14.512550 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.514141 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.516120 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.516864 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.517036 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.523413 (+ 6377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:14.523433 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:14.523456 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:14.523536 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:14.528777 (+ 5241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.529896 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.529909 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.529913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.530117 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.530587 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.530592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.530594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.530987 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:14.531385 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.531759 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.532096 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:44:14.533886 30003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:14.639246 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 29865
I20260812 02:44:14.667425 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 30015
Aug 12 02:44:14 dist-test-slave-ss4n krb5kdc[29743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:44:14.685726 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.668382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.17.196.194:41817)
0812 02:44:14.668604 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.668609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.668630 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:44:14.668699 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.668703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.668778 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.668870 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:14.670388 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.671332 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.672924 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.673563 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.673697 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.678565 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:14.678583 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:14.678602 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:14.678649 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:14.682360 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.683194 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.683203 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.683205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.683356 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.683932 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.683937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.683939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.684431 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:14.684759 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.685098 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.685300 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:44:14.686522 30157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:14.782612 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 30015
I20260812 02:44:14.804327 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 30170
Aug 12 02:44:14 dist-test-slave-ss4n krb5kdc[29743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:44:14.822440 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:44:14.805234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54060 (local address 127.17.196.195:40623)
0812 02:44:14.805505 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 02:44:14.805512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:44:14.805538 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:44:14.805790 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:44:14.805797 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:44:14.805878 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:44:14.805999 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:44:14.807334 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.808183 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.809659 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:44:14.810093 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:44:14.810254 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:44:14.815442 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:44:14.815463 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:44:14.815482 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:44:14.815535 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:44:14.819119 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.819890 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.819898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.819901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.820101 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:44:14.820610 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:44:14.820614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:44:14.820616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:44:14.821015 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:44:14.821295 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 02:44:14.821814 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 02:44:14.822012 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:44:14.823189 30311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:14.918481 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 30170
I20260812 02:44:14.940472 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 29759
I20260812 02:44:14.941938 29829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:44:15.057546 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 29759
2026-08-12T02:44:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4888 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:44:15 dist-test-slave-ss4n krb5kdc[30378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:44:15 dist-test-slave-ss4n krb5kdc[30378](info): set up 2 sockets
Aug 12 02:44:15 dist-test-slave-ss4n krb5kdc[30378](info): commencing operation
krb5kdc: starting...
W20260812 02:44:16.339135 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:44:16 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:44:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:44:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/rangeradmin_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/rangeradmin_127.17.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/rangerlookup_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/rangerlookup_127.17.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/HTTP_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/HTTP_127.17.196.212@KRBTEST.COM.keytab.
I20260812 02:44:16.809418 18195 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDYHVZq/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/postgres -l logfile start
2026-08-12 02:44:19.832 UTC [30411] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 02:44:19.832 UTC [30411] LOG: listening on IPv4 address "127.17.196.212", port 43497
2026-08-12 02:44:19.837 UTC [30411] LOG: listening on Unix socket "/tmp/.s.PGSQL.43497"
2026-08-12 02:44:19.864 UTC [30414] LOG: database system was shut down at 2026-08-12 02:44:17 UTC
2026-08-12 02:44:19.869 UTC [30411] LOG: database system is ready to accept connections
I20260812 02:44:20.958207 18195 mini_postgres.cc:96] Postgres bound to 43497
2026-08-12 02:44:20.984 UTC [30424] FATAL: database "slave" does not exist
127.17.196.212:43497 - accepting connections
I20260812 02:44:20.985123 18195 mini_ranger.cc:161] Starting Ranger...
I20260812 02:44:21.020561 18195 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 02:44:21.673804 18195 mini_ranger.cc:87] Created ranger Postgres database
I20260812 02:44:21.673971 18195 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-admin
2026-08-12 02:44:22,025 [I] DB FLAVOR :POSTGRES
2026-08-12 02:44:22,026 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 02:44:22,026 [I] Checking connection..
2026-08-12 02:44:22,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 02:44:22,307 [I] Checking connection passed.
2026-08-12 02:44:22,308 [I] --------- Verifying version history table ---------
2026-08-12 02:44:22,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:44:22,579 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 02:44:22,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:44:22,875 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 02:44:22,875 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 02:44:22,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 02:44:23.154 UTC [30501] WARNING: there is no transaction in progress
2026-08-12 02:44:23,163 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 02:44:23,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:44:23,427 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 02:44:23,428 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 02:44:23,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 02:44:23,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 02:44:23,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-12 02:44:24,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 02:44:24,490 [I] Table x_portal_user does not exist in database ranger
2026-08-12 02:44:24,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 02:44:24,770 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 02:44:24,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 02:44:25,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 02:44:25,304 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 02:44:25,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 02:44:26.476 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.496 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.515 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.534 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.558 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.583 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.653 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.662 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.681 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.694 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.706 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.725 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.737 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.745 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.754 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:26.763 UTC [30624] WARNING: there is no transaction in progress
2026-08-12 02:44:27,584 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 02:44:27,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 02:44:27,865 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 02:44:27,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 02:44:28,147 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 02:44:28,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 02:44:28,409 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 02:44:28,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 02:44:28,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 02:44:28,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-12 02:44:29,224 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 02:44:29,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 02:44:29,505 [I] DB_PATCHES have already been applied
I20260812 02:44:29.511909 18195 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDYHVZq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 02:44:29.512030 18195 mini_ranger.cc:193] Using host: 127.17.196.212
I20260812 02:44:29.535619 18195 mini_ranger.cc:227] Ranger admin URL: http://127.17.196.212:56103
Aug 12, 2026 2:44:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:44:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:44:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:44:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:44:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:44:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:44:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 02:44:30.809181 18195 mini_ranger.cc:160] Time spent starting Ranger: real 9.824s user 0.004s sys 0.072s
Aug 12, 2026 2:44:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:44:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 2:44:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:44:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:44:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 02:44:54.767961 18195 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/rangerkms_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/rangerkms_127.17.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.196.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.196.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.196.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/HTTP_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.196.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/HTTP_127.17.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 02:44:54 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502694, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 02:44:55.001178 18195 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 02:44:55.034219 18195 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 02:44:56.109622 18195 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 02:44:56.109788 18195 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms
I20260812 02:44:56.109941 18195 mini_ranger_kms.cc:227] Using postgres at 127.17.196.212:43497
2026-08-12 02:44:56,832 [I] DB FLAVOR :POSTGRES
2026-08-12 02:44:56,833 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 02:44:56,833 [I] Checking connection
2026-08-12 02:44:56,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 02:44:57,113 [I] connection success
2026-08-12 02:44:57,113 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 02:44:57,113 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 02:44:57,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 02:44:57,377 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 02:44:57,377 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 02:44:57,377 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 02:44:57,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:43497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 02:44:57,674 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 02:44:58.200929 18195 mini_ranger_kms.cc:310] Created kms service
I20260812 02:44:58.414307 18195 mini_ranger_kms.cc:326] Created kudu user
I20260812 02:44:58.467793 18195 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 02:44:58 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502694, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:44:58.912091 18195 mini_ranger_kms.cc:384] Added ranger policy
I20260812 02:44:58.913890 18195 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 02:44:58.914005 18195 mini_ranger_kms.cc:240] Using host: 127.17.196.212
I20260812 02:44:58.947510 18195 mini_ranger_kms.cc:276] Ranger KMS PID: 30835
I20260812 02:44:58.947724 18195 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.196.212:46815
Aug 12, 2026 2:45:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:45:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:45:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:45:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:45:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:45:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 02:45:01.925799 18195 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.925s user 0.143s sys 0.483s
I20260812 02:45:01.926180 18195 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 02:45:01.926434 18195 mini_ranger_kms.cc:401] 127.17.196.212:46815/kms/v1/keys
Aug 12 02:45:02 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.196.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 02:45:02 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.196.212@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
Aug 12 02:45:02 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.196.212@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
Aug 12 02:45:05 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:45:05.706719 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:33427
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:33427
--ranger_config_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:46815/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:45:06.312692 30901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:45:06.313091 30901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:45:06.313370 30901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:45:06.323082 30901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:45:06.323354 30901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:45:06.323510 30901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:45:06.323643 30901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:45:06.323741 30901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:45:06.337392 30901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.196.212:46815/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:33427
--ranger_config_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:33427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:45:06.338936 30901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:45:06.340557 30901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:45:06.357177 30906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:45:06.358165 30909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:45:06.358165 30907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:45:06.359879 30901 server_base.cc:1034] running on GCE node
I20260812 02:45:06.360808 30901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:45:06.362619 30901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:45:06.363881 30901 hybrid_clock.cc:648] HybridClock initialized: now 1786502706363823 us; error 49 us; skew 500 ppm
I20260812 02:45:06.364428 30901 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:45:06 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502706, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:45:06.369272 30901 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:45:06.371806 30901 webserver.cc:460] Webserver started at http://127.17.196.254:33423/ using document root <none> and password file <none>
I20260812 02:45:06.372742 30901 fs_manager.cc:359] Metadata directory not provided
I20260812 02:45:06.373013 30901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:45:06.373438 30901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:45:06 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502706, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:45:06.434119 30901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "50b0ac4d0a8845edaee64287fe549a6a"
format_stamp: "Formatted at 2026-08-12 02:45:06 on dist-test-slave-ss4n"
server_key: "115eab4fca207c04f3c107d016a74eda"
server_key_iv: "d0ac5bb7c316911be1f7d0b91a1a98ba"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:06.435101 30901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "50b0ac4d0a8845edaee64287fe549a6a"
format_stamp: "Formatted at 2026-08-12 02:45:06 on dist-test-slave-ss4n"
server_key: "115eab4fca207c04f3c107d016a74eda"
server_key_iv: "d0ac5bb7c316911be1f7d0b91a1a98ba"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:06.442605 30901 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260812 02:45:06.463693 30916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:06.465943 30901 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260812 02:45:06.466244 30901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "50b0ac4d0a8845edaee64287fe549a6a"
format_stamp: "Formatted at 2026-08-12 02:45:06 on dist-test-slave-ss4n"
server_key: "115eab4fca207c04f3c107d016a74eda"
server_key_iv: "d0ac5bb7c316911be1f7d0b91a1a98ba"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:06.466609 30901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:45:06.506321 30901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:45:06.511828 30901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:45:06.512379 30901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:45:06.548199 30901 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:33427
I20260812 02:45:06.548441 30982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:33427 every 8 connection(s)
I20260812 02:45:06.550057 30901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:45:06.555368 30983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:45:06.556140 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 30901
I20260812 02:45:06.556403 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:45:06.557271 18195 external_mini_cluster.cc:1428] Setting key 3b748165e00a562ed9eb2dfa3c8d64f0
I20260812 02:45:06.577096 30983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a: Bootstrap starting.
Aug 12 02:45:06 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:45:06.582880 30983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a: Neither blocks nor log segments found. Creating new log.
I20260812 02:45:06.584474 30983 log.cc:824] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:45:06.589425 30983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a: No bootstrap required, opened a new log
I20260812 02:45:06.590230 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:06.559818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.17.196.254:33427)
0812 02:45:06.561747 (+ 1929us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:06.561760 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:06.561831 (+ 71us) server_negotiation.cc:407] Connection header received
0812 02:45:06.565027 (+ 3196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:06.565064 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:06.565648 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:06.566109 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:45:06.568328 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:06.570154 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:06.572469 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:06.573793 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:06.573929 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:06.580367 (+ 6438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:45:06.580377 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:45:06.580394 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:45:06.580439 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:45:06.584920 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:06.586163 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:06.586175 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:06.586194 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:06.586440 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:06.586984 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:06.586989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:06.586994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:06.587618 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:45:06.587965 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:06.588495 (+ 530us) server_negotiation.cc:299] Negotiation successful
0812 02:45:06.588938 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1555,"thread_start_us":993,"threads_started":1}
I20260812 02:45:06.599776 30983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0ac4d0a8845edaee64287fe549a6a" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33427 } }
I20260812 02:45:06.600528 30983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:45:06.600685 30983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50b0ac4d0a8845edaee64287fe549a6a, State: Initialized, Role: FOLLOWER
I20260812 02:45:06.601449 30983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0ac4d0a8845edaee64287fe549a6a" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33427 } }
I20260812 02:45:06.601934 30983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:45:06.602123 30983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:45:06.602360 30983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:45:06.605468 30983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0ac4d0a8845edaee64287fe549a6a" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33427 } }
I20260812 02:45:06.606227 30983 leader_election.cc:302] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50b0ac4d0a8845edaee64287fe549a6a; no voters:
I20260812 02:45:06.607045 30983 leader_election.cc:288] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:45:06.607275 30989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:45:06.609541 30989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [term 1 LEADER]: Becoming Leader. State: Replica: 50b0ac4d0a8845edaee64287fe549a6a, State: Running, Role: LEADER
I20260812 02:45:06.610469 30983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:45:06.610426 30989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0ac4d0a8845edaee64287fe549a6a" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33427 } }
I20260812 02:45:06.614794 30991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50b0ac4d0a8845edaee64287fe549a6a. Latest consensus state: current_term: 1 leader_uuid: "50b0ac4d0a8845edaee64287fe549a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0ac4d0a8845edaee64287fe549a6a" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33427 } } }
I20260812 02:45:06.614924 30990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50b0ac4d0a8845edaee64287fe549a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0ac4d0a8845edaee64287fe549a6a" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 33427 } } }
I20260812 02:45:06.615581 30991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [sys.catalog]: This master's current role is: LEADER
I20260812 02:45:06.615597 30990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a [sys.catalog]: This master's current role is: LEADER
I20260812 02:45:06.620150 30996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:45:06.628446 30996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:45:06.629525 30983 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 02:45:06.654403 30996 catalog_manager.cc:1347] Generated new cluster ID: ddcdc0f52fbf406293c87e752542b80e
I20260812 02:45:06.654603 30996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:45:06.691582 30996 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:45:06.693702 30996 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:45:06.709758 30996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a: Generated new TSK 0
I20260812 02:45:06.711298 30996 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:45:07.610855 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:46815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33427
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:45:07 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502707, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 02:45:08.276239 31039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:45:08.276961 31039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:45:08.277328 31039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:45:08.291972 31039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:45:08.292336 31039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:45:08.292694 31039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:45:08.308744 31039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.196.212:46815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:45:08.310599 31039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:45:08.312717 31039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:45:08.341650 31052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:45:08.341666 31053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:45:08.344700 31055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:45:08.347684 31039 server_base.cc:1034] running on GCE node
I20260812 02:45:08.349221 31039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:45:08.351291 31039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:45:08.352685 31039 hybrid_clock.cc:648] HybridClock initialized: now 1786502708352570 us; error 84 us; skew 500 ppm
I20260812 02:45:08.353332 31039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:45:08 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502708, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:45:08.361047 31039 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:45:08.364568 31039 webserver.cc:460] Webserver started at http://127.17.196.193:35099/ using document root <none> and password file <none>
I20260812 02:45:08.365726 31039 fs_manager.cc:359] Metadata directory not provided
I20260812 02:45:08.365979 31039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:45:08.366505 31039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:45:08 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502708, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:45:08.387683 31039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "81825e945a384563bcfd29414c7d401e"
format_stamp: "Formatted at 2026-08-12 02:45:08 on dist-test-slave-ss4n"
server_key: "e2ecf304527a265bd3ab8ae06fd3b146"
server_key_iv: "6ab637ac6c62a47f9709c652d5e2b7fb"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:08.388893 31039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "81825e945a384563bcfd29414c7d401e"
format_stamp: "Formatted at 2026-08-12 02:45:08 on dist-test-slave-ss4n"
server_key: "e2ecf304527a265bd3ab8ae06fd3b146"
server_key_iv: "6ab637ac6c62a47f9709c652d5e2b7fb"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:08.397119 31039 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260812 02:45:08.438814 31065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:08.441635 31039 fs_manager.cc:725] Time spent opening block manager: real 0.026s user 0.008s sys 0.015s
I20260812 02:45:08.442034 31039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "81825e945a384563bcfd29414c7d401e"
format_stamp: "Formatted at 2026-08-12 02:45:08 on dist-test-slave-ss4n"
server_key: "e2ecf304527a265bd3ab8ae06fd3b146"
server_key_iv: "6ab637ac6c62a47f9709c652d5e2b7fb"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:08.442512 31039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:45:08.527395 31039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:45:08.536229 31039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:45:08.536873 31039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:45:08.539048 31039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:45:08.541965 31039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:45:08.542101 31039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:08.542300 31039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:45:08.542510 31039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 12 02:45:08 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502707, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:45:08.725628 31039 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:43001
I20260812 02:45:08.725701 31196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:43001 every 8 connection(s)
I20260812 02:45:08.727227 31039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:45:08.733879 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 31039
I20260812 02:45:08.734146 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:45:08.735215 18195 external_mini_cluster.cc:1428] Setting key c8c6d92e78500c71f981a0ca45f99b6c
Aug 12 02:45:08 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502708, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
I20260812 02:45:08.811511 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:08.732895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:40927 (local address 127.17.196.254:33427)
0812 02:45:08.766692 (+ 33797us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:08.766703 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:08.771981 (+ 5278us) server_negotiation.cc:407] Connection header received
0812 02:45:08.783383 (+ 11402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:08.783390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:08.783516 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:08.783675 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:45:08.787075 (+ 3400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:08.788728 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:08.792482 (+ 3754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:08.793511 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:08.793648 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:08.800129 (+ 6481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:45:08.800141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:45:08.800149 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:45:08.800204 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:45:08.806575 (+ 6371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:08.807818 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:08.807824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:08.807828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:08.808005 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:08.808765 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:08.808771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:08.808775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:08.809263 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:45:08.809405 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:08.810635 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0812 02:45:08.810936 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33577,"thread_start_us":33797,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
I20260812 02:45:08.814558 31201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:08.731723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33427 (local address 127.17.196.193:40927)
0812 02:45:08.768115 (+ 36392us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:08.768257 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:08.782991 (+ 14734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:08.784361 (+ 1370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:08.784375 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:08.785412 (+ 1037us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:45:08.786798 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:08.786833 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:08.789098 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:08.789106 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:08.792250 (+ 3144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:08.792283 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:08.794205 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:08.794215 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:08.794666 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:08.795703 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:45:08.795740 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:45:08.799649 (+ 3909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:45:08.806907 (+ 7258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:45:08.806920 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:45:08.806943 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:45:08.807644 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:45:08.808148 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:45:08.808154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:45:08.808158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:45:08.808613 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:45:08.809543 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:45:08.809552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:45:08.810143 (+ 591us) client_negotiation.cc:769] Sending connection context
0812 02:45:08.811871 (+ 1728us) client_negotiation.cc:240] Negotiation successful
0812 02:45:08.812650 (+ 779us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35804,"thread_start_us":42578,"threads_started":1}
I20260812 02:45:08.817297 31197 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33427
I20260812 02:45:08.817757 31197 heartbeater.cc:464] Registering TS with master...
I20260812 02:45:08.819065 31197 heartbeater.cc:511] Master 127.17.196.254:33427 requested a full tablet report, sending...
I20260812 02:45:08.822336 30937 ts_manager.cc:194] Registered new tserver with Master: 81825e945a384563bcfd29414c7d401e (127.17.196.193:43001)
I20260812 02:45:08.824690 30937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:40927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
W20260812 02:45:08.858616 18195 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.196.194: real 0.120s user 0.000s sys 0.029s
I20260812 02:45:08.859547 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:46815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33427
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:45:09 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502707, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
W20260812 02:45:09.790232 31205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:45:09.790845 31205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:45:09.791164 31205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:45:09.807907 31205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:45:09.808343 31205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:45:09.808717 31205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:45:09.830588 31197 heartbeater.cc:503] Master 127.17.196.254:33427 was elected leader, sending a full tablet report...
I20260812 02:45:09.833333 31205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.196.212:46815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:45:09.835673 31205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:45:09.838009 31205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:45:09.858704 31221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:45:09.859606 31222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:45:09.859753 31224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:45:09.860673 31205 server_base.cc:1034] running on GCE node
I20260812 02:45:09.861459 31205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:45:09.862849 31205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:45:09.864163 31205 hybrid_clock.cc:648] HybridClock initialized: now 1786502709863986 us; error 95 us; skew 500 ppm
I20260812 02:45:09.864639 31205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:45:09 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502709, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:45:09.869719 31205 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:45:09.872308 31205 webserver.cc:460] Webserver started at http://127.17.196.194:44997/ using document root <none> and password file <none>
I20260812 02:45:09.873311 31205 fs_manager.cc:359] Metadata directory not provided
I20260812 02:45:09.873607 31205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:45:09.874068 31205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:45:09 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502709, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:45:09.894574 31205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "5976da6aa6b74c06b50c1427c596f148"
format_stamp: "Formatted at 2026-08-12 02:45:09 on dist-test-slave-ss4n"
server_key: "35997a2dfc058361e0bf85846cfb1b3a"
server_key_iv: "c39cd8e4d6844cf8e0f1e56c9c9f3144"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:09.896435 31205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "5976da6aa6b74c06b50c1427c596f148"
format_stamp: "Formatted at 2026-08-12 02:45:09 on dist-test-slave-ss4n"
server_key: "35997a2dfc058361e0bf85846cfb1b3a"
server_key_iv: "c39cd8e4d6844cf8e0f1e56c9c9f3144"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:09.907222 31205 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.010s sys 0.002s
I20260812 02:45:09.931531 31231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:09.933465 31205 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:45:09.933815 31205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "5976da6aa6b74c06b50c1427c596f148"
format_stamp: "Formatted at 2026-08-12 02:45:09 on dist-test-slave-ss4n"
server_key: "35997a2dfc058361e0bf85846cfb1b3a"
server_key_iv: "c39cd8e4d6844cf8e0f1e56c9c9f3144"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:09.934196 31205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:45:09.957978 31205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:45:09.964970 31205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:45:09.965813 31205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:45:09.967888 31205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:45:09.970340 31205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:45:09.970546 31205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:09.970785 31205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:45:09.970932 31205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:10.069309 31205 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:36767
I20260812 02:45:10.069404 31361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:36767 every 8 connection(s)
I20260812 02:45:10.071163 31205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:45:10.075346 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 31205
I20260812 02:45:10.075567 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:45:10.076207 18195 external_mini_cluster.cc:1428] Setting key 1fb35007d62fa94bca95afae46d13110
Aug 12 02:45:10 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502709, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:45:10.104204 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:10.077156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:42031 (local address 127.17.196.254:33427)
0812 02:45:10.078472 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:10.078479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:10.078660 (+ 181us) server_negotiation.cc:407] Connection header received
0812 02:45:10.081678 (+ 3018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:10.081684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:10.081773 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:10.081894 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:45:10.084222 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:10.085716 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:10.088159 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:10.088814 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:10.089030 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:10.094951 (+ 5921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:45:10.094958 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:45:10.094962 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:45:10.095033 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:45:10.099416 (+ 4383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:10.100494 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:10.100505 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:10.100508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:10.100772 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:10.101388 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:10.101395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:10.101398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:10.101960 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:45:10.102197 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:10.103165 (+ 968us) server_negotiation.cc:299] Negotiation successful
0812 02:45:10.103458 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1157,"thread_start_us":1591,"threads_started":1}
I20260812 02:45:10.105762 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:10.076543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33427 (local address 127.17.196.194:42031)
0812 02:45:10.078458 (+ 1915us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:10.078527 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:10.081343 (+ 2816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:10.082230 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:10.082239 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:10.082986 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:45:10.083940 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:10.083962 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:10.085882 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:10.085891 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:10.087863 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:10.087907 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:10.089224 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:10.089234 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:10.089719 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:10.090844 (+ 1125us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:45:10.090869 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:45:10.094581 (+ 3712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:45:10.099611 (+ 5030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:45:10.099623 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:45:10.099645 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:45:10.100338 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:45:10.100900 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:45:10.100906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:45:10.100910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:45:10.101252 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:45:10.102179 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:45:10.102193 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:45:10.102712 (+ 519us) client_negotiation.cc:769] Sending connection context
0812 02:45:10.103070 (+ 358us) client_negotiation.cc:240] Negotiation successful
0812 02:45:10.103886 (+ 816us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1523,"thread_start_us":987,"threads_started":1}
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
I20260812 02:45:10.108124 31362 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33427
I20260812 02:45:10.108525 31362 heartbeater.cc:464] Registering TS with master...
I20260812 02:45:10.109855 31362 heartbeater.cc:511] Master 127.17.196.254:33427 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
I20260812 02:45:10.111872 30937 ts_manager.cc:194] Registered new tserver with Master: 5976da6aa6b74c06b50c1427c596f148 (127.17.196.194:36767)
I20260812 02:45:10.112991 30937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:42031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
W20260812 02:45:10.198782 18195 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.196.195: real 0.119s user 0.003s sys 0.019s
I20260812 02:45:10.200021 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:46815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33427
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:45:10.943621 31368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:45:10.944042 31368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:45:10.944249 31368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:45:10.955854 31368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:45:10.956074 31368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:45:10.956295 31368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:45:10.970956 31368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.196.212:46815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:33427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:45:10.972867 31368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:45:10.974515 31368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:45:10.995759 31374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:45:10.995721 31376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:45:10.995857 31373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:45:10.997359 31368 server_base.cc:1034] running on GCE node
I20260812 02:45:10.998327 31368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:45:10.999821 31368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:45:11.001210 31368 hybrid_clock.cc:648] HybridClock initialized: now 1786502711001057 us; error 53 us; skew 500 ppm
I20260812 02:45:11.001811 31368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:45:11 dist-test-slave-ss4n krb5kdc[30378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502711, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:45:11.012161 31368 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:45:11.014662 31368 webserver.cc:460] Webserver started at http://127.17.196.195:43613/ using document root <none> and password file <none>
I20260812 02:45:11.015558 31368 fs_manager.cc:359] Metadata directory not provided
I20260812 02:45:11.015758 31368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:45:11.016208 31368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:45:11 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502711, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:45:11.035640 31368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1"
format_stamp: "Formatted at 2026-08-12 02:45:11 on dist-test-slave-ss4n"
server_key: "869795eccb90495d6ccd24f95920bfda"
server_key_iv: "a837d16445a3dcdc53c333f411bb87a9"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:11.036976 31368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1"
format_stamp: "Formatted at 2026-08-12 02:45:11 on dist-test-slave-ss4n"
server_key: "869795eccb90495d6ccd24f95920bfda"
server_key_iv: "a837d16445a3dcdc53c333f411bb87a9"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:11.045797 31368 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.011s sys 0.000s
I20260812 02:45:11.067219 31385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:11.069303 31368 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:45:11.069628 31368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1"
format_stamp: "Formatted at 2026-08-12 02:45:11 on dist-test-slave-ss4n"
server_key: "869795eccb90495d6ccd24f95920bfda"
server_key_iv: "a837d16445a3dcdc53c333f411bb87a9"
server_key_version: "kuduclusterkey@0"
I20260812 02:45:11.069936 31368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:45:11.093971 31368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:45:11.100268 31368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:45:11.100870 31368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:45:11.102514 31368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:45:11.105136 31368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:45:11.105225 31368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:11.105409 31368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:45:11.105571 31368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:45:11.116487 31362 heartbeater.cc:503] Master 127.17.196.254:33427 was elected leader, sending a full tablet report...
I20260812 02:45:11.202893 31368 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:42093
I20260812 02:45:11.203059 31516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:42093 every 8 connection(s)
I20260812 02:45:11.204989 31368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:45:11.207515 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 31368
I20260812 02:45:11.207741 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:45:11.208350 18195 external_mini_cluster.cc:1428] Setting key acbdbfc6e1ba637746e70ed3730a95f0
Aug 12 02:45:11 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502711, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:45:11.238106 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.209169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:49119 (local address 127.17.196.254:33427)
0812 02:45:11.210235 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.210242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.212760 (+ 2518us) server_negotiation.cc:407] Connection header received
0812 02:45:11.216123 (+ 3363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.216128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.216202 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.216302 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:45:11.218919 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.220362 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.223170 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.223921 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.224118 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.229192 (+ 5074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:45:11.229197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:45:11.229201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:45:11.229263 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:45:11.233681 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:11.234733 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:11.234738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:11.234740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:11.234931 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:11.235558 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:11.235562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:11.235564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:11.235972 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:45:11.236194 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.237156 (+ 962us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.237424 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":759,"threads_started":1}
I20260812 02:45:11.239516 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.210198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:33427 (local address 127.17.196.195:49119)
0812 02:45:11.212118 (+ 1920us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:11.212170 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:11.215744 (+ 3574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:11.216655 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:11.216667 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:11.217597 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:45:11.218655 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.218679 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.220612 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.220617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.222872 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.222920 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.224349 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.224359 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.224669 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.225638 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:45:11.225664 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:45:11.228847 (+ 3183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:45:11.233869 (+ 5022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:45:11.233888 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:45:11.233904 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:45:11.234560 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:45:11.235061 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:45:11.235066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:45:11.235070 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:45:11.235410 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:45:11.236123 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:45:11.236156 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:45:11.236607 (+ 451us) client_negotiation.cc:769] Sending connection context
0812 02:45:11.236952 (+ 345us) client_negotiation.cc:240] Negotiation successful
0812 02:45:11.237841 (+ 889us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1462,"thread_start_us":842,"threads_started":1}
I20260812 02:45:11.241189 31517 heartbeater.cc:347] Connected to a master server at 127.17.196.254:33427
I20260812 02:45:11.241600 31517 heartbeater.cc:464] Registering TS with master...
I20260812 02:45:11.242679 31517 heartbeater.cc:511] Master 127.17.196.254:33427 requested a full tablet report, sending...
I20260812 02:45:11.244500 30937 ts_manager.cc:194] Registered new tserver with Master: a02bcfccff9f43ae835c9f9b6bbf2af1 (127.17.196.195:42093)
I20260812 02:45:11.245431 30937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:49119
I20260812 02:45:11.252705 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:45:11.283221 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.260360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.17.196.254:33427)
0812 02:45:11.260700 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.260712 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.261699 (+ 987us) server_negotiation.cc:407] Connection header received
0812 02:45:11.261950 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.261955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.262057 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.262169 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:45:11.264544 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.265985 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.268163 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.269096 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.269218 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.273698 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:45:11.273705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:45:11.273709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:45:11.273753 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:45:11.278800 (+ 5047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:11.280051 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:11.280057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:11.280059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:11.280288 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:11.280872 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:11.280877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:11.280879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:11.281640 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:45:11.282006 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.282409 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.282719 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260812 02:45:11.289788 30937 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:45:11.302733 30937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:45:11.345758 31531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.324057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:36767 (local address 127.0.0.1:58448)
0812 02:45:11.329943 (+ 5886us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:11.330022 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:11.330344 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:11.331606 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:11.331618 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:11.331666 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:45:11.333679 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.333697 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.336665 (+ 2968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.336681 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.340734 (+ 4053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.340750 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.344309 (+ 3559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.344316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.344616 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.344675 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 02:45:11.344851 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 02:45:11.345105 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5433,"thread_start_us":5076,"threads_started":1}
I20260812 02:45:11.346099 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.324662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58448 (local address 127.17.196.194:36767)
0812 02:45:11.326635 (+ 1973us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.326645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.330671 (+ 4026us) server_negotiation.cc:407] Connection header received
0812 02:45:11.330823 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.330833 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.331258 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.331578 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:45:11.333983 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.336492 (+ 2509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.341021 (+ 4529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.343657 (+ 2636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.344051 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.344284 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.344954 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.345212 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1742,"spinlock_wait_cycles":26496,"thread_start_us":1647,"threads_started":1}
I20260812 02:45:11.347615 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.326076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.17.196.193:43001)
0812 02:45:11.328271 (+ 2195us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.328283 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.333639 (+ 5356us) server_negotiation.cc:407] Connection header received
0812 02:45:11.334056 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.334062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.334246 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.334461 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:45:11.336004 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.338644 (+ 2640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.342973 (+ 4329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.345154 (+ 2181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.345587 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.345797 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.346729 (+ 932us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.346984 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1821,"thread_start_us":1462,"threads_started":1}
I20260812 02:45:11.347784 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.324980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43001 (local address 127.0.0.1:41332)
0812 02:45:11.333382 (+ 8402us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:11.333440 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:11.333801 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:11.334452 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:11.334457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:11.334482 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:45:11.335717 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.335730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.338842 (+ 3112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.338847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.342680 (+ 3833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.342696 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.346149 (+ 3453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.346159 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.346439 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.346491 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 02:45:11.346641 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 02:45:11.346826 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8164,"mutex_wait_us":23,"thread_start_us":7641,"threads_started":1}
I20260812 02:45:11.347828 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.325989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.17.196.195:42093)
0812 02:45:11.330772 (+ 4783us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.330780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.332892 (+ 2112us) server_negotiation.cc:407] Connection header received
0812 02:45:11.333029 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.333035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.333347 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.333593 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:45:11.336914 (+ 3321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.339228 (+ 2314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.343000 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.345234 (+ 2234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.345571 (+ 337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.345865 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.346955 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.347226 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4550,"thread_start_us":4219,"threads_started":1}
I20260812 02:45:11.348738 31533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.324847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:42093 (local address 127.0.0.1:41054)
0812 02:45:11.331790 (+ 6943us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:11.331810 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:11.332053 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:11.333940 (+ 1887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:11.333947 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:11.333971 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:45:11.335672 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.335691 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.339402 (+ 3711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.339407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.342786 (+ 3379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.342802 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.346266 (+ 3464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.346281 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.346533 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.346558 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:45:11.346679 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 02:45:11.346817 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6719,"mutex_wait_us":3,"thread_start_us":6392,"threads_started":1}
I20260812 02:45:11.352690 31287 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5433d452f94c26a908cc14a67e0407 (DEFAULT_TABLE table=test-table [id=309c6c88f32b4db0949bc57720ea2176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:45:11.353319 31441 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5433d452f94c26a908cc14a67e0407 (DEFAULT_TABLE table=test-table [id=309c6c88f32b4db0949bc57720ea2176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:45:11.353943 31121 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5433d452f94c26a908cc14a67e0407 (DEFAULT_TABLE table=test-table [id=309c6c88f32b4db0949bc57720ea2176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:45:11.355132 31287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5433d452f94c26a908cc14a67e0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:45:11.355978 31441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5433d452f94c26a908cc14a67e0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:45:11.356199 31121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5433d452f94c26a908cc14a67e0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:45:11.375974 31537 tablet_bootstrap.cc:492] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Bootstrap starting.
I20260812 02:45:11.380185 31538 tablet_bootstrap.cc:492] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Bootstrap starting.
I20260812 02:45:11.382251 31539 tablet_bootstrap.cc:492] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Bootstrap starting.
I20260812 02:45:11.384613 31537 tablet_bootstrap.cc:654] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Neither blocks nor log segments found. Creating new log.
I20260812 02:45:11.387450 31537 log.cc:824] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Log is configured to *not* fsync() on all Append() calls
I20260812 02:45:11.388062 31538 tablet_bootstrap.cc:654] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Neither blocks nor log segments found. Creating new log.
I20260812 02:45:11.388340 31539 tablet_bootstrap.cc:654] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Neither blocks nor log segments found. Creating new log.
I20260812 02:45:11.391000 31539 log.cc:824] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:45:11.391269 31538 log.cc:824] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Log is configured to *not* fsync() on all Append() calls
I20260812 02:45:11.395349 31537 tablet_bootstrap.cc:492] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: No bootstrap required, opened a new log
I20260812 02:45:11.395968 31537 ts_tablet_manager.cc:1397] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.004s
I20260812 02:45:11.397713 31539 tablet_bootstrap.cc:492] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: No bootstrap required, opened a new log
I20260812 02:45:11.398257 31539 ts_tablet_manager.cc:1397] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.003s
I20260812 02:45:11.399629 31538 tablet_bootstrap.cc:492] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: No bootstrap required, opened a new log
I20260812 02:45:11.400189 31538 ts_tablet_manager.cc:1397] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.016s
I20260812 02:45:11.410969 31539 raft_consensus.cc:348] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.411536 31539 raft_consensus.cc:374] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:45:11.411737 31539 raft_consensus.cc:729] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81825e945a384563bcfd29414c7d401e, State: Initialized, Role: FOLLOWER
I20260812 02:45:11.412642 31539 consensus_queue.cc:260] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.414196 31537 raft_consensus.cc:348] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.415184 31537 raft_consensus.cc:374] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:45:11.415462 31537 raft_consensus.cc:729] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02bcfccff9f43ae835c9f9b6bbf2af1, State: Initialized, Role: FOLLOWER
I20260812 02:45:11.415958 31539 ts_tablet_manager.cc:1428] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Time spent starting tablet: real 0.017s user 0.005s sys 0.011s
I20260812 02:45:11.416532 31537 consensus_queue.cc:260] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.418807 31517 heartbeater.cc:503] Master 127.17.196.254:33427 was elected leader, sending a full tablet report...
I20260812 02:45:11.418644 31538 raft_consensus.cc:348] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.419394 31538 raft_consensus.cc:374] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:45:11.419694 31538 raft_consensus.cc:729] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5976da6aa6b74c06b50c1427c596f148, State: Initialized, Role: FOLLOWER
I20260812 02:45:11.420470 31538 consensus_queue.cc:260] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.422020 31537 ts_tablet_manager.cc:1428] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Time spent starting tablet: real 0.026s user 0.017s sys 0.007s
I20260812 02:45:11.424935 31538 ts_tablet_manager.cc:1428] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Time spent starting tablet: real 0.025s user 0.012s sys 0.011s
W20260812 02:45:11.458128 31518 tablet.cc:2369] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:45:11.511205 31545 raft_consensus.cc:482] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:45:11.511716 31545 raft_consensus.cc:504] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.514536 31545 leader_election.cc:288] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5976da6aa6b74c06b50c1427c596f148 (127.17.196.194:36767), 81825e945a384563bcfd29414c7d401e (127.17.196.193:43001)
I20260812 02:45:11.521195 31544 raft_consensus.cc:482] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:45:11.521822 31544 raft_consensus.cc:504] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.525831 31544 leader_election.cc:288] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5976da6aa6b74c06b50c1427c596f148 (127.17.196.194:36767), a02bcfccff9f43ae835c9f9b6bbf2af1 (127.17.196.195:42093)
I20260812 02:45:11.526575 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.515269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:38357 (local address 127.17.196.193:43001)
0812 02:45:11.515591 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.515599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.515634 (+ 35us) server_negotiation.cc:407] Connection header received
0812 02:45:11.515766 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.515771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.515859 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.516010 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:45:11.517078 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.518289 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.521867 (+ 3578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.524022 (+ 2155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.524292 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.524423 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.525909 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.526071 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:45:11.527287 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.514620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43001 (local address 127.17.196.195:38357)
0812 02:45:11.515043 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:11.515067 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:11.515276 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:11.516088 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:11.516096 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:11.516246 (+ 150us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:45:11.516859 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.516880 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.518478 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.518482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.521609 (+ 3127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.521626 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.524669 (+ 3043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.524676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.525000 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.525040 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:45:11.525146 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 02:45:11.525354 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260812 02:45:11.529973 31144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5433d452f94c26a908cc14a67e0407" candidate_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81825e945a384563bcfd29414c7d401e" is_pre_election: true
I20260812 02:45:11.530787 31144 raft_consensus.cc:2454] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02bcfccff9f43ae835c9f9b6bbf2af1 in term 0.
W20260812 02:45:11.531917 31200 tablet.cc:2369] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:45:11.533573 31386 leader_election.cc:302] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81825e945a384563bcfd29414c7d401e, a02bcfccff9f43ae835c9f9b6bbf2af1; no voters:
I20260812 02:45:11.534389 31545 raft_consensus.cc:2781] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:45:11.534747 31545 raft_consensus.cc:482] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:45:11.534945 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.515389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:48105 (local address 127.17.196.194:36767)
0812 02:45:11.515668 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.515675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.518255 (+ 2580us) server_negotiation.cc:407] Connection header received
0812 02:45:11.518456 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.518462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.518549 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.518624 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:45:11.520438 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.522270 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.529841 (+ 7571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.532601 (+ 2760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.532847 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.532975 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.534168 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.534396 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:45:11.534970 31545 raft_consensus.cc:3037] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:45:11.539520 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.515920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:36767 (local address 127.17.196.195:48105)
0812 02:45:11.518099 (+ 2179us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:11.518121 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:11.518289 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:11.518708 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:11.518714 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:11.518742 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:45:11.520169 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.520187 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.522519 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.522525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.529437 (+ 6912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.529529 (+ 92us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.533357 (+ 3828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.533368 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.533875 (+ 507us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.533939 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 02:45:11.538128 (+ 4189us) client_negotiation.cc:240] Negotiation successful
0812 02:45:11.538398 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1897,"thread_start_us":1592,"threads_started":1}
I20260812 02:45:11.540488 31545 raft_consensus.cc:504] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.541383 31308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5433d452f94c26a908cc14a67e0407" candidate_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5976da6aa6b74c06b50c1427c596f148" is_pre_election: true
I20260812 02:45:11.542829 31144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5433d452f94c26a908cc14a67e0407" candidate_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81825e945a384563bcfd29414c7d401e"
I20260812 02:45:11.542923 31308 raft_consensus.cc:2454] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02bcfccff9f43ae835c9f9b6bbf2af1 in term 0.
I20260812 02:45:11.543315 31144 raft_consensus.cc:3037] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:45:11.543074 31307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5433d452f94c26a908cc14a67e0407" candidate_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5976da6aa6b74c06b50c1427c596f148"
I20260812 02:45:11.543777 31307 raft_consensus.cc:3037] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:45:11.550602 31144 raft_consensus.cc:2454] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02bcfccff9f43ae835c9f9b6bbf2af1 in term 1.
I20260812 02:45:11.551020 31545 leader_election.cc:288] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [CANDIDATE]: Term 1 election: Requested vote from peers 5976da6aa6b74c06b50c1427c596f148 (127.17.196.194:36767), 81825e945a384563bcfd29414c7d401e (127.17.196.193:43001)
I20260812 02:45:11.551889 31386 leader_election.cc:302] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81825e945a384563bcfd29414c7d401e, a02bcfccff9f43ae835c9f9b6bbf2af1; no voters:
I20260812 02:45:11.552384 31545 raft_consensus.cc:2781] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:45:11.552416 31307 raft_consensus.cc:2454] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02bcfccff9f43ae835c9f9b6bbf2af1 in term 1.
I20260812 02:45:11.553905 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.525013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:49013 (local address 127.17.196.194:36767)
0812 02:45:11.535297 (+ 10284us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.535306 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.535333 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:45:11.535413 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.535421 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.535543 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.537342 (+ 1799us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:45:11.538067 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.540191 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.546745 (+ 6554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.551375 (+ 4630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.551668 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.551759 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.553255 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.553414 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10131,"thread_start_us":12933,"threads_started":1}
I20260812 02:45:11.554385 31545 raft_consensus.cc:686] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 1 LEADER]: Becoming Leader. State: Replica: a02bcfccff9f43ae835c9f9b6bbf2af1, State: Running, Role: LEADER
I20260812 02:45:11.555624 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.525005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:36767 (local address 127.17.196.193:49013)
0812 02:45:11.528929 (+ 3924us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:11.528951 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:11.529145 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:11.535754 (+ 6609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:11.535763 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:11.535804 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:45:11.537579 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.537602 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.540505 (+ 2903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.540513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.546428 (+ 5915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.546452 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.552467 (+ 6015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.552476 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.552765 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.552803 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 02:45:11.552900 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 02:45:11.553066 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3722,"mutex_wait_us":418,"thread_start_us":4770,"threads_started":1}
I20260812 02:45:11.555401 31545 consensus_queue.cc:237] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } }
I20260812 02:45:11.556560 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.526714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:35495 (local address 127.17.196.195:42093)
0812 02:45:11.526985 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.526991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.531573 (+ 4582us) server_negotiation.cc:407] Connection header received
0812 02:45:11.531997 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.532003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.532129 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.532207 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:45:11.534489 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.536284 (+ 1795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.546586 (+ 10302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.549060 (+ 2474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.549345 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.549504 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.555932 (+ 6428us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.556185 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:45:11.556643 31307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5433d452f94c26a908cc14a67e0407" candidate_uuid: "81825e945a384563bcfd29414c7d401e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5976da6aa6b74c06b50c1427c596f148" is_pre_election: true
I20260812 02:45:11.557228 31307 raft_consensus.cc:2379] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 81825e945a384563bcfd29414c7d401e in current term 1: Already voted for candidate a02bcfccff9f43ae835c9f9b6bbf2af1 in this term.
I20260812 02:45:11.557981 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.526713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:42093 (local address 127.17.196.193:35495)
0812 02:45:11.531454 (+ 4741us) negotiation.cc:107] Waiting for socket to connect
0812 02:45:11.531485 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 02:45:11.531732 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:45:11.532545 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:45:11.532550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:45:11.532577 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:45:11.534195 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.534213 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.536936 (+ 2723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.536941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.546217 (+ 9276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:45:11.546259 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.555271 (+ 9012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.555297 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:45:11.555683 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.555735 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 02:45:11.556804 (+ 1069us) client_negotiation.cc:240] Negotiation successful
0812 02:45:11.557074 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4481,"mutex_wait_us":95,"thread_start_us":1632,"threads_started":1}
I20260812 02:45:11.559891 31466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5433d452f94c26a908cc14a67e0407" candidate_uuid: "81825e945a384563bcfd29414c7d401e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" is_pre_election: true
I20260812 02:45:11.562353 31069 leader_election.cc:302] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81825e945a384563bcfd29414c7d401e; no voters: 5976da6aa6b74c06b50c1427c596f148, a02bcfccff9f43ae835c9f9b6bbf2af1
I20260812 02:45:11.563424 31544 raft_consensus.cc:2726] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 02:45:11.565647 30935 catalog_manager.cc:5495] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a02bcfccff9f43ae835c9f9b6bbf2af1 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a02bcfccff9f43ae835c9f9b6bbf2af1" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42093 } health_report { overall_health: HEALTHY } } }
W20260812 02:45:11.575634 31363 tablet.cc:2369] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:45:11.593758 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.585272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.17.196.195:42093)
0812 02:45:11.585579 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.585585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.585697 (+ 112us) server_negotiation.cc:407] Connection header received
0812 02:45:11.585978 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.585982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.586066 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.586236 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:45:11.586806 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.588140 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.590427 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.591043 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.591200 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.592098 (+ 898us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:45:11.592812 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:45:11.593042 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.593098 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.593185 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:45:11.604719 31144 raft_consensus.cc:1261] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 1 FOLLOWER]: Refusing update from remote peer a02bcfccff9f43ae835c9f9b6bbf2af1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:45:11.604663 31307 raft_consensus.cc:1261] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 1 FOLLOWER]: Refusing update from remote peer a02bcfccff9f43ae835c9f9b6bbf2af1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:45:11.606280 31551 consensus_queue.cc:1035] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5976da6aa6b74c06b50c1427c596f148" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 36767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 02:45:11.607064 31545 consensus_queue.cc:1035] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81825e945a384563bcfd29414c7d401e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:45:11.627655 31558 mvcc.cc:205] Tried to move back new op lower bound from 7317515106718978048 to 7317515106542878720. Current Snapshot: MvccSnapshot[applied={T|T < 7317515106718978048}]
I20260812 02:45:11.650290 30935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:45:11.650862 30935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:45:11.656739 30935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50b0ac4d0a8845edaee64287fe549a6a: Sending DeleteTablet for 3 replicas of tablet 0e5433d452f94c26a908cc14a67e0407
I20260812 02:45:11.658686 31121 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e5433d452f94c26a908cc14a67e0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:45:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:41332
I20260812 02:45:11.658640 31287 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e5433d452f94c26a908cc14a67e0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:45:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:58448
I20260812 02:45:11.658908 31441 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e5433d452f94c26a908cc14a67e0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:45:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:41054
I20260812 02:45:11.662637 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 31039
I20260812 02:45:11.674633 31568 tablet_replica.cc:330] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: stopping tablet replica
I20260812 02:45:11.675782 31570 tablet_replica.cc:330] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: stopping tablet replica
I20260812 02:45:11.676151 31568 raft_consensus.cc:2229] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:45:11.676555 31570 raft_consensus.cc:2229] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:45:11.677318 31568 raft_consensus.cc:2258] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:45:11.677434 31570 raft_consensus.cc:2258] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:45:11.680092 31569 tablet_replica.cc:330] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: stopping tablet replica
I20260812 02:45:11.680245 31570 ts_tablet_manager.cc:1905] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:45:11.680576 31568 ts_tablet_manager.cc:1905] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:45:11.680577 31569 raft_consensus.cc:2229] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:45:11.681145 31569 raft_consensus.cc:2258] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:45:11.685798 31569 ts_tablet_manager.cc:1905] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:45:11 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:45:11.696038 31568 ts_tablet_manager.cc:1918] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:45:11.696348 31568 log.cc:1196] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/0e5433d452f94c26a908cc14a67e0407
I20260812 02:45:11.697091 31569 ts_tablet_manager.cc:1918] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:45:11.697347 31569 log.cc:1196] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/0e5433d452f94c26a908cc14a67e0407
I20260812 02:45:11.697506 31568 ts_tablet_manager.cc:1939] T 0e5433d452f94c26a908cc14a67e0407 P 81825e945a384563bcfd29414c7d401e: Deleting consensus metadata
I20260812 02:45:11.698319 31569 ts_tablet_manager.cc:1939] T 0e5433d452f94c26a908cc14a67e0407 P 5976da6aa6b74c06b50c1427c596f148: Deleting consensus metadata
I20260812 02:45:11.700505 30918 catalog_manager.cc:4841] TS 81825e945a384563bcfd29414c7d401e (127.17.196.193:43001): tablet 0e5433d452f94c26a908cc14a67e0407 (table test-table [id=309c6c88f32b4db0949bc57720ea2176]) successfully deleted
I20260812 02:45:11.700843 30919 catalog_manager.cc:4841] TS 5976da6aa6b74c06b50c1427c596f148 (127.17.196.194:36767): tablet 0e5433d452f94c26a908cc14a67e0407 (table test-table [id=309c6c88f32b4db0949bc57720ea2176]) successfully deleted
I20260812 02:45:11.701169 31570 ts_tablet_manager.cc:1918] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:45:11.701634 31570 log.cc:1196] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/0e5433d452f94c26a908cc14a67e0407
I20260812 02:45:11.702673 31570 ts_tablet_manager.cc:1939] T 0e5433d452f94c26a908cc14a67e0407 P a02bcfccff9f43ae835c9f9b6bbf2af1: Deleting consensus metadata
I20260812 02:45:11.703979 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.665322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41338 (local address 127.17.196.193:43001)
0812 02:45:11.665629 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.665639 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.678668 (+ 13029us) server_negotiation.cc:407] Connection header received
0812 02:45:11.678918 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.678924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.679020 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.679154 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:45:11.681590 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.682751 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.684909 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.685634 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.685756 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.692994 (+ 7238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:45:11.693037 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:45:11.693060 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:45:11.693120 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:45:11.698414 (+ 5294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:11.699913 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:11.699924 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:11.699929 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:11.700133 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:11.700714 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:11.700720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:11.700724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:11.701706 (+ 982us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:45:11.702087 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.703003 (+ 916us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.703289 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 02:45:11.704998 31191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:45:11.705629 30921 catalog_manager.cc:4841] TS a02bcfccff9f43ae835c9f9b6bbf2af1 (127.17.196.195:42093): tablet 0e5433d452f94c26a908cc14a67e0407 (table test-table [id=309c6c88f32b4db0949bc57720ea2176]) successfully deleted
I20260812 02:45:11.812537 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 31039
I20260812 02:45:11.836679 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 31205
Aug 12 02:45:11 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:45:11.857753 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.837744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58458 (local address 127.17.196.194:36767)
0812 02:45:11.837934 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.837939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.837958 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:45:11.838124 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.838128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.838217 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.838299 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:45:11.840211 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.841140 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.843300 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.843776 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.843875 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:11.849796 (+ 5921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:45:11.849818 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:45:11.849842 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:45:11.849897 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:45:11.854136 (+ 4239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:11.855038 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:11.855049 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:11.855053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:11.855229 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:11.855742 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:11.855748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:11.855753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:11.856242 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:45:11.856630 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:11.857068 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 02:45:11.857336 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:45:11.858739 31355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:45:11.962668 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 31205
I20260812 02:45:11.986421 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 31368
Aug 12 02:45:11 dist-test-slave-ss4n krb5kdc[30378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:45:12.007290 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:45:11.987435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.17.196.195:42093)
0812 02:45:11.987718 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 02:45:11.987725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:45:11.987832 (+ 107us) server_negotiation.cc:407] Connection header received
0812 02:45:11.988081 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:45:11.988086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:45:11.988267 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:45:11.988441 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:45:11.990371 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.991207 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.993285 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:45:11.993971 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:45:11.994100 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:45:12.000104 (+ 6004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:45:12.000118 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:45:12.000134 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:45:12.000176 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:45:12.004203 (+ 4027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:12.005184 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:12.005192 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:12.005194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:12.005328 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:45:12.005792 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:45:12.005797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:45:12.005800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:45:12.006236 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:45:12.006550 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 02:45:12.006866 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 02:45:12.007052 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 02:45:12.008158 31512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:45:12.135519 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 31368
I20260812 02:45:12.159456 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 30901
I20260812 02:45:12.160928 30977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:45:12.278672 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 30901
I20260812 02:45:12.306787 18195 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 02:45:12.653462 18195 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 02:45:12.653906 18195 mini_ranger.cc:66] Stopping Ranger...
I20260812 02:45:13.001001 18195 mini_ranger.cc:68] Stopped Ranger
2026-08-12 02:45:13.001 UTC [30411] LOG: received smart shutdown request
2026-08-12 02:45:13.006 UTC [30411] LOG: background worker "logical replication launcher" (PID 30422) exited with exit code 1
2026-08-12 02:45:13.006 UTC [30416] LOG: shutting down
2026-08-12 02:45:13.226 UTC [30411] LOG: database system is shut down
2026-08-12T02:45:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (58318 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:45:13 dist-test-slave-ss4n krb5kdc[31587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:45:13 dist-test-slave-ss4n krb5kdc[31587](info): set up 2 sockets
Aug 12 02:45:13 dist-test-slave-ss4n krb5kdc[31587](info): commencing operation
krb5kdc: starting...
W20260812 02:45:14.626335 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:45:14 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:45:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/rangeradmin_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/rangeradmin_127.17.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/rangerlookup_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/rangerlookup_127.17.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/HTTP_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/HTTP_127.17.196.212@KRBTEST.COM.keytab.
I20260812 02:45:15.067082 18195 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDYHVZq/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/postgres -l logfile start
2026-08-12 02:45:18.112 UTC [31621] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 02:45:18.112 UTC [31621] LOG: listening on IPv4 address "127.17.196.212", port 36787
2026-08-12 02:45:18.116 UTC [31621] LOG: listening on Unix socket "/tmp/.s.PGSQL.36787"
2026-08-12 02:45:18.141 UTC [31625] LOG: database system was shut down at 2026-08-12 02:45:16 UTC
2026-08-12 02:45:18.146 UTC [31621] LOG: database system is ready to accept connections
I20260812 02:45:19.248351 18195 mini_postgres.cc:96] Postgres bound to 36787
2026-08-12 02:45:19.272 UTC [31633] FATAL: database "slave" does not exist
127.17.196.212:36787 - accepting connections
I20260812 02:45:19.273375 18195 mini_ranger.cc:161] Starting Ranger...
I20260812 02:45:19.307893 18195 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 02:45:19.946946 18195 mini_ranger.cc:87] Created ranger Postgres database
I20260812 02:45:19.947093 18195 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-admin
2026-08-12 02:45:20,263 [I] DB FLAVOR :POSTGRES
2026-08-12 02:45:20,263 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 02:45:20,264 [I] Checking connection..
2026-08-12 02:45:20,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 02:45:20,534 [I] Checking connection passed.
2026-08-12 02:45:20,534 [I] --------- Verifying version history table ---------
2026-08-12 02:45:20,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:45:20,814 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 02:45:20,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:45:21,084 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 02:45:21,084 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 02:45:21,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 02:45:21.350 UTC [31708] WARNING: there is no transaction in progress
2026-08-12 02:45:21,358 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 02:45:21,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:45:21,628 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 02:45:21,628 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 02:45:21,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 02:45:21,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 02:45:22,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-12 02:45:22,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 02:45:22,697 [I] Table x_portal_user does not exist in database ranger
2026-08-12 02:45:22,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 02:45:22,952 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 02:45:22,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 02:45:23,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 02:45:23,501 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 02:45:23,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 02:45:24.583 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.602 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.621 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.639 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.658 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.683 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.751 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.759 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.776 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.788 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.801 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.816 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.828 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.835 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.844 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:24.852 UTC [31832] WARNING: there is no transaction in progress
2026-08-12 02:45:25,686 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 02:45:25,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 02:45:25,962 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 02:45:25,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 02:45:26,249 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 02:45:26,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 02:45:26,518 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 02:45:26,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 02:45:26,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 02:45:27,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-12 02:45:27,301 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 02:45:27,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 02:45:27,568 [I] DB_PATCHES have already been applied
I20260812 02:45:27.574663 18195 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDYHVZq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 02:45:27.574806 18195 mini_ranger.cc:193] Using host: 127.17.196.212
I20260812 02:45:27.595899 18195 mini_ranger.cc:227] Ranger admin URL: http://127.17.196.212:41919
Aug 12, 2026 2:45:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:45:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 02:45:28.912578 18195 mini_ranger.cc:160] Time spent starting Ranger: real 9.639s user 0.003s sys 0.061s
Aug 12, 2026 2:45:49 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:45:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 2:45:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:45:49 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:45:50 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 02:45:51.128403 18195 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/rangerkms_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/rangerkms_127.17.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.196.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.196.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.196.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/HTTP_127.17.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.196.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/HTTP_127.17.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 02:45:51 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502751, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 02:45:51.350212 18195 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 02:45:51.378340 18195 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 02:45:52.470099 18195 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 02:45:52.470242 18195 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms
I20260812 02:45:52.470309 18195 mini_ranger_kms.cc:227] Using postgres at 127.17.196.212:36787
2026-08-12 02:45:52,866 [I] DB FLAVOR :POSTGRES
2026-08-12 02:45:52,866 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 02:45:52,866 [I] Checking connection
2026-08-12 02:45:52,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 02:45:53,133 [I] connection success
2026-08-12 02:45:53,134 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 02:45:53,134 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 02:45:53,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 02:45:53,407 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 02:45:53,408 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 02:45:53,408 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 02:45:53,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.196.212:36787/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 02:45:53,690 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 02:45:54.235456 18195 mini_ranger_kms.cc:310] Created kms service
I20260812 02:45:54.468946 18195 mini_ranger_kms.cc:326] Created kudu user
I20260812 02:45:54.519868 18195 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 02:45:54 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502751, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:45:54.965837 18195 mini_ranger_kms.cc:384] Added ranger policy
I20260812 02:45:54.966192 18195 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDYHVZq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDYHVZq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 02:45:54.966267 18195 mini_ranger_kms.cc:240] Using host: 127.17.196.212
I20260812 02:45:54.987860 18195 mini_ranger_kms.cc:276] Ranger KMS PID: 32051
I20260812 02:45:54.988005 18195 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.196.212:52991
Aug 12, 2026 2:45:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:45:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:45:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:45:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:45:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:45:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 02:45:57.487679 18195 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.137s user 0.047s sys 0.377s
I20260812 02:45:57.487982 18195 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 02:45:57.488169 18195 mini_ranger_kms.cc:401] 127.17.196.212:52991/kms/v1/keys
Aug 12 02:45:57 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.196.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 02:45:58 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.196.212@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
Aug 12 02:45:58 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.196.212@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
Aug 12 02:46:00 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:46:01.042538 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44729
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44729
--ranger_config_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:52991/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:01.402624 32108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:01.402933 32108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:01.403023 32108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:01.410364 32108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:01.410492 32108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:01.410547 32108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:01.410605 32108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:01.410656 32108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:01.421537 32108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.196.212:52991/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44729
--ranger_config_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:01.422940 32108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:01.424263 32108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:01.437070 32114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:01.437158 32118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:01.437158 32115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:01.437991 32108 server_base.cc:1034] running on GCE node
I20260812 02:46:01.438865 32108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:01.440455 32108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:01.441684 32108 hybrid_clock.cc:648] HybridClock initialized: now 1786502761441644 us; error 42 us; skew 500 ppm
I20260812 02:46:01.442065 32108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:01 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:01.451795 32108 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:46:01.453790 32108 webserver.cc:460] Webserver started at http://127.17.196.254:33525/ using document root <none> and password file <none>
I20260812 02:46:01.454493 32108 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:01.454609 32108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:01.454905 32108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:46:01 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:01.520761 32108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "1bd59b4409a74fdcb542f0e9830e19a4"
format_stamp: "Formatted at 2026-08-12 02:46:01 on dist-test-slave-ss4n"
server_key: "dacd8695fe505ad3d41006848d5dc116"
server_key_iv: "d767cecfe013f55a071155ffeeb89fa8"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:01.521971 32108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "1bd59b4409a74fdcb542f0e9830e19a4"
format_stamp: "Formatted at 2026-08-12 02:46:01 on dist-test-slave-ss4n"
server_key: "dacd8695fe505ad3d41006848d5dc116"
server_key_iv: "d767cecfe013f55a071155ffeeb89fa8"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:01.529623 32108 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260812 02:46:01.550215 32127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:01.552130 32108 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:46:01.552345 32108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "1bd59b4409a74fdcb542f0e9830e19a4"
format_stamp: "Formatted at 2026-08-12 02:46:01 on dist-test-slave-ss4n"
server_key: "dacd8695fe505ad3d41006848d5dc116"
server_key_iv: "d767cecfe013f55a071155ffeeb89fa8"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:01.552613 32108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:01.591099 32108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:01.596681 32108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:01.597054 32108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:01.619827 32108 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:44729
I20260812 02:46:01.619841 32184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:44729 every 8 connection(s)
I20260812 02:46:01.621657 32108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:46:01.626075 32185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:01.627374 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 32108
I20260812 02:46:01.627638 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:46:01.628577 18195 external_mini_cluster.cc:1428] Setting key f0e7acbfd47a70f9fe3a2caea777eb3c
I20260812 02:46:01.645663 32185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Bootstrap starting.
I20260812 02:46:01.650347 32185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:01.651782 32185 log.cc:824] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:01.656894 32185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: No bootstrap required, opened a new log
Aug 12 02:46:01 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:01.667533 32185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } }
I20260812 02:46:01.668089 32185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:01.668233 32185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd59b4409a74fdcb542f0e9830e19a4, State: Initialized, Role: FOLLOWER
I20260812 02:46:01.669066 32185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } }
I20260812 02:46:01.669385 32185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:46:01.669579 32185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:46:01.669806 32185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:01.672634 32185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } }
I20260812 02:46:01.672807 32189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:01.630982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.17.196.254:44729)
0812 02:46:01.645756 (+ 14774us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:01.645774 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:01.645840 (+ 66us) server_negotiation.cc:407] Connection header received
0812 02:46:01.648534 (+ 2694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:01.648569 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:01.649154 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:01.649716 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:01.651432 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:01.653978 (+ 2546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:01.655950 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:01.657024 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:01.658050 (+ 1026us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:01.662877 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:01.662888 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:01.662916 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:01.662979 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:01.667336 (+ 4357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:01.668445 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:01.668455 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:01.668464 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:01.668674 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:01.669173 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:01.669178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:01.669181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:01.670002 (+ 821us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:01.670316 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:01.670848 (+ 532us) server_negotiation.cc:299] Negotiation successful
0812 02:46:01.671325 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14427,"thread_start_us":14820,"threads_started":1}
I20260812 02:46:01.673288 32185 leader_election.cc:302] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bd59b4409a74fdcb542f0e9830e19a4; no voters:
I20260812 02:46:01.674077 32185 leader_election.cc:288] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:46:01.674214 32191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:01.678788 32191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 1 LEADER]: Becoming Leader. State: Replica: 1bd59b4409a74fdcb542f0e9830e19a4, State: Running, Role: LEADER
I20260812 02:46:01.679569 32191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } }
I20260812 02:46:01.679960 32185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:01.690927 32194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bd59b4409a74fdcb542f0e9830e19a4. Latest consensus state: current_term: 1 leader_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } } }
I20260812 02:46:01.691515 32194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:01.704152 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } } }
I20260812 02:46:01.704612 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:01.705698 32203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:46:01.715821 32203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:46:01.716727 32185 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 02:46:01.746948 32203 catalog_manager.cc:1347] Generated new cluster ID: 6731f524f77f43b5a9a963bb36a1e4aa
I20260812 02:46:01.747493 32203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:46:01.851847 32203 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:46:01.854060 32203 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:46:01.866277 32203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Generated new TSK 0
I20260812 02:46:01.867518 32203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:46:02.614319 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:52991/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44729
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:46:02 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502762, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 02:46:03.072190 32240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:03.072716 32240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:03.072955 32240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:03.084972 32240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:03.085140 32240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:03.085389 32240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:46:03.101502 32240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.196.212:52991/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:03.103657 32240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:03.105592 32240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:03.128207 32250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:03.130448 32252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:03.145594 32255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:03.146893 32240 server_base.cc:1034] running on GCE node
I20260812 02:46:03.147852 32240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:03.149307 32240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:03.150660 32240 hybrid_clock.cc:648] HybridClock initialized: now 1786502763150548 us; error 107 us; skew 500 ppm
I20260812 02:46:03.151144 32240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:03 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502763, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:03.155171 32240 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:46:03.157472 32240 webserver.cc:460] Webserver started at http://127.17.196.193:35323/ using document root <none> and password file <none>
I20260812 02:46:03.158470 32240 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:03.158645 32240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:03.159060 32240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:46:03 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502763, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:03.178995 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "8983ed99b4bd42d588ca874b0ba7b260"
format_stamp: "Formatted at 2026-08-12 02:46:03 on dist-test-slave-ss4n"
server_key: "5206228402a7a4db33aea8f1bf96ff72"
server_key_iv: "fd7df03f40a74ad03c6772f0b5e2d233"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:03.180176 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "8983ed99b4bd42d588ca874b0ba7b260"
format_stamp: "Formatted at 2026-08-12 02:46:03 on dist-test-slave-ss4n"
server_key: "5206228402a7a4db33aea8f1bf96ff72"
server_key_iv: "fd7df03f40a74ad03c6772f0b5e2d233"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:03.201961 32240 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.014s sys 0.008s
I20260812 02:46:03.222802 32263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:03.224820 32240 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:46:03.225033 32240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "8983ed99b4bd42d588ca874b0ba7b260"
format_stamp: "Formatted at 2026-08-12 02:46:03 on dist-test-slave-ss4n"
server_key: "5206228402a7a4db33aea8f1bf96ff72"
server_key_iv: "fd7df03f40a74ad03c6772f0b5e2d233"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:03.225327 32240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:03.248262 32240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:03.254182 32240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:03.254627 32240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:03.256099 32240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:03.258347 32240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:03.258466 32240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:03.258625 32240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:03.258699 32240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:03.362835 32240 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:42739
I20260812 02:46:03.362854 32401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:42739 every 8 connection(s)
I20260812 02:46:03.364797 32240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:46:03.373797 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 32240
I20260812 02:46:03.374059 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:46:03.374902 18195 external_mini_cluster.cc:1428] Setting key 782c08ae288d8ef1198482db95bcd558
Aug 12 02:46:03 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502762, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
Aug 12 02:46:03 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502763, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:03.409502 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:03.369152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44729 (local address 127.17.196.193:33559)
0812 02:46:03.374229 (+ 5077us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:03.374296 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:03.387141 (+ 12845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:03.388016 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:03.388027 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:03.388554 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:03.389442 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:03.389467 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:03.391220 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:03.391227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:03.393465 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:03.393512 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:03.394898 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:03.394907 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:03.395239 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:03.396183 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:03.396218 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:03.399692 (+ 3474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:03.404107 (+ 4415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:03.404118 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:03.404139 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:03.404815 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:03.405352 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:03.405357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:03.405361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:03.405724 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:03.406519 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:03.406529 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:03.407022 (+ 493us) client_negotiation.cc:769] Sending connection context
0812 02:46:03.407360 (+ 338us) client_negotiation.cc:240] Negotiation successful
0812 02:46:03.407970 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4639,"thread_start_us":7354,"threads_started":1}
I20260812 02:46:03.410938 32405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:03.370148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:33559 (local address 127.17.196.254:44729)
0812 02:46:03.386656 (+ 16508us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:03.386663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:03.386694 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:46:03.387490 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:03.387496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:03.387596 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:03.387715 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:03.389751 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:03.391001 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:03.393744 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:03.394397 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:03.394546 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:03.400039 (+ 5493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:03.400047 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:03.400052 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:03.400110 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:03.403895 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:03.404992 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:03.405005 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:03.405008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:03.405197 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:03.405880 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:03.405884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:03.405887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:03.406260 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:03.406424 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:03.410279 (+ 3855us) server_negotiation.cc:299] Negotiation successful
0812 02:46:03.410550 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16364,"thread_start_us":16239,"threads_started":1}
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
I20260812 02:46:03.413803 32402 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44729
I20260812 02:46:03.414202 32402 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:03.415247 32402 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
I20260812 02:46:03.418285 32145 ts_manager.cc:194] Registered new tserver with Master: 8983ed99b4bd42d588ca874b0ba7b260 (127.17.196.193:42739)
I20260812 02:46:03.420482 32145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:33559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:46:03.462270 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:52991/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44729
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:46:03 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502762, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
W20260812 02:46:03.995440 32408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:03.995994 32408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:03.996198 32408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:04.008122 32408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:04.008383 32408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:04.008666 32408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:46:04.024266 32408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.196.212:52991/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:04.025925 32408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:04.027279 32408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:04.045742 32415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:04.046213 32416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:04.059569 32419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:04.066691 32408 server_base.cc:1034] running on GCE node
I20260812 02:46:04.067689 32408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:04.076815 32408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:04.081190 32408 hybrid_clock.cc:648] HybridClock initialized: now 1786502764081019 us; error 164 us; skew 500 ppm
I20260812 02:46:04.081745 32408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:04 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502764, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:04.086737 32408 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:46:04.094789 32408 webserver.cc:460] Webserver started at http://127.17.196.194:34067/ using document root <none> and password file <none>
I20260812 02:46:04.095824 32408 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:04.096004 32408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:04.096381 32408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:46:04 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502764, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:04.116744 32408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "a4e38fafa85b47a79b851383e14ede87"
format_stamp: "Formatted at 2026-08-12 02:46:04 on dist-test-slave-ss4n"
server_key: "266d33030161313bf6df933dfffaab56"
server_key_iv: "981bed87723f21981b4ac9f7fdff172c"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:04.118021 32408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "a4e38fafa85b47a79b851383e14ede87"
format_stamp: "Formatted at 2026-08-12 02:46:04 on dist-test-slave-ss4n"
server_key: "266d33030161313bf6df933dfffaab56"
server_key_iv: "981bed87723f21981b4ac9f7fdff172c"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:04.125814 32408 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260812 02:46:04.165252 32436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:04.168005 32408 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.009s sys 0.002s
I20260812 02:46:04.168380 32408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "a4e38fafa85b47a79b851383e14ede87"
format_stamp: "Formatted at 2026-08-12 02:46:04 on dist-test-slave-ss4n"
server_key: "266d33030161313bf6df933dfffaab56"
server_key_iv: "981bed87723f21981b4ac9f7fdff172c"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:04.168772 32408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:04.216890 32408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:04.222698 32408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:04.223119 32408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:04.231101 32408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:04.240162 32408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:04.240294 32408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:04.240430 32408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:04.240496 32408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:04.281330 32408 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:40557
I20260812 02:46:04.281458 32569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:40557 every 8 connection(s)
I20260812 02:46:04.283092 32408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:46:04.290241 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 32408
I20260812 02:46:04.290462 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:46:04.291113 18195 external_mini_cluster.cc:1428] Setting key 0c4719292b4b1b11dcf5b917d5d0817c
Aug 12 02:46:04 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502764, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:04.307511 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:04.286232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:47923 (local address 127.17.196.254:44729)
0812 02:46:04.286828 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:04.286835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:04.288175 (+ 1340us) server_negotiation.cc:407] Connection header received
0812 02:46:04.290822 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:04.290829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:04.290951 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:04.291080 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:04.293157 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:04.294132 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:04.296036 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:04.296514 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:04.296725 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:04.300658 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:04.300664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:04.300667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:04.300703 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:04.303863 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:04.304771 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:04.304778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:04.304782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:04.304930 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:04.305458 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:04.305467 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:04.305470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:04.305829 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:04.306009 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:04.306854 (+ 845us) server_negotiation.cc:299] Negotiation successful
0812 02:46:04.307118 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":308,"threads_started":1}
I20260812 02:46:04.309024 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:04.286964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44729 (local address 127.17.196.194:47923)
0812 02:46:04.287972 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:04.288024 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:04.290280 (+ 2256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:04.291392 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:04.291404 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:04.292052 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:04.292913 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:04.292940 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:04.294328 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:04.294334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:04.295852 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:04.295875 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:04.296831 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:04.296840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:04.297111 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:04.297939 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:04.297966 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:04.300394 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:04.304040 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:04.304051 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:04.304071 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:04.304629 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:04.305062 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:04.305068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:04.305072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:04.305329 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:04.305996 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:04.306006 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:04.306390 (+ 384us) client_negotiation.cc:769] Sending connection context
0812 02:46:04.306727 (+ 337us) client_negotiation.cc:240] Negotiation successful
0812 02:46:04.307391 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":374,"threads_started":1}
I20260812 02:46:04.310811 32570 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44729
I20260812 02:46:04.311174 32570 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:04.312045 32570 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
I20260812 02:46:04.313843 32145 ts_manager.cc:194] Registered new tserver with Master: a4e38fafa85b47a79b851383e14ede87 (127.17.196.194:40557)
I20260812 02:46:04.314954 32145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:47923
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:46:04.366315 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:52991/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44729
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:46:04.424705 32402 heartbeater.cc:503] Master 127.17.196.254:44729 was elected leader, sending a full tablet report...
W20260812 02:46:04.737914 32577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:04.738226 32577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:04.738327 32577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:04.746129 32577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:04.746275 32577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:04.746392 32577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:46:04.756686 32577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.196.212:52991/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:44729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:04.758266 32577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:04.759824 32577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:04.774551 32583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:04.774688 32585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:04.774706 32582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:04.780200 32577 server_base.cc:1034] running on GCE node
I20260812 02:46:04.781273 32577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:04.782598 32577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:04.783890 32577 hybrid_clock.cc:648] HybridClock initialized: now 1786502764783808 us; error 87 us; skew 500 ppm
I20260812 02:46:04.784296 32577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:04 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502764, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:04.788462 32577 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:46:04.790311 32577 webserver.cc:460] Webserver started at http://127.17.196.195:46827/ using document root <none> and password file <none>
I20260812 02:46:04.791090 32577 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:04.791208 32577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:04.791500 32577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:46:04 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502764, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:04.809579 32577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "0e45c2f653fd4713bf5a2b18357f64be"
format_stamp: "Formatted at 2026-08-12 02:46:04 on dist-test-slave-ss4n"
server_key: "9e059bf67ff68d86cb6c10f515def4c1"
server_key_iv: "61f6b71e7d4b887e761bf23dac8fba70"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:04.810674 32577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "0e45c2f653fd4713bf5a2b18357f64be"
format_stamp: "Formatted at 2026-08-12 02:46:04 on dist-test-slave-ss4n"
server_key: "9e059bf67ff68d86cb6c10f515def4c1"
server_key_iv: "61f6b71e7d4b887e761bf23dac8fba70"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:04.817376 32577 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 02:46:04.835877 32592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:04.837872 32577 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 02:46:04.838105 32577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "0e45c2f653fd4713bf5a2b18357f64be"
format_stamp: "Formatted at 2026-08-12 02:46:04 on dist-test-slave-ss4n"
server_key: "9e059bf67ff68d86cb6c10f515def4c1"
server_key_iv: "61f6b71e7d4b887e761bf23dac8fba70"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:04.838385 32577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:04.859779 32577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:04.864916 32577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:04.865307 32577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:04.866593 32577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:04.868235 32577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:04.868350 32577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:04.868523 32577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:04.868609 32577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:04.933118 32577 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:41151
I20260812 02:46:04.933208 32731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:41151 every 8 connection(s)
I20260812 02:46:04.935567 32577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:46:04.942538 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 32577
I20260812 02:46:04.942852 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:46:04.943902 18195 external_mini_cluster.cc:1428] Setting key b42fb1dc55dca7ace1463adf3ff4deeb
Aug 12 02:46:04 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502764, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:04.971161 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:04.940890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:41341 (local address 127.17.196.254:44729)
0812 02:46:04.942029 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:04.942034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:04.944048 (+ 2014us) server_negotiation.cc:407] Connection header received
0812 02:46:04.948183 (+ 4135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:04.948195 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:04.948335 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:04.948584 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:04.950920 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:04.952183 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:04.954666 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:04.955588 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:04.955718 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:04.962236 (+ 6518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:04.962242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:04.962246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:04.962314 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:04.966889 (+ 4575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:04.967884 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:04.967890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:04.967892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:04.968104 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:04.968793 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:04.968797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:04.968800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:04.969152 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:04.969326 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:04.970254 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 02:46:04.970551 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":801,"threads_started":1}
I20260812 02:46:04.972553 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:04.941872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44729 (local address 127.17.196.195:41341)
0812 02:46:04.943789 (+ 1917us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:04.943845 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:04.947777 (+ 3932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:04.948877 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:04.948888 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:04.949634 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:04.950697 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:04.950722 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:04.952433 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:04.952437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:04.954454 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:04.954498 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:04.955948 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:04.955954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:04.956318 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:04.957317 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:04.957344 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:04.961834 (+ 4490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:04.967095 (+ 5261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:04.967112 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:04.967129 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:04.967709 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:04.968240 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:04.968246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:04.968249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:04.968642 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:04.969366 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:04.969376 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:04.969796 (+ 420us) client_negotiation.cc:769] Sending connection context
0812 02:46:04.970143 (+ 347us) client_negotiation.cc:240] Negotiation successful
0812 02:46:04.970903 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1585,"thread_start_us":1183,"threads_started":1}
I20260812 02:46:04.975217 32732 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44729
I20260812 02:46:04.975864 32732 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:04.977301 32732 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
I20260812 02:46:04.980062 32144 ts_manager.cc:194] Registered new tserver with Master: 0e45c2f653fd4713bf5a2b18357f64be (127.17.196.195:41151)
I20260812 02:46:04.981966 32144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:41341
I20260812 02:46:04.990618 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:46:04.992472 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 32240
Aug 12 02:46:05 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:46:05.022149 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:04.993864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39476 (local address 127.17.196.193:42739)
0812 02:46:04.995076 (+ 1212us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:04.995084 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:04.995113 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:46:04.995179 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:04.995185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:04.995447 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:04.995600 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:04.998359 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:05.000245 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:05.003108 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:05.004230 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:05.004380 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:05.011951 (+ 7571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:05.012022 (+ 71us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:05.012042 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:05.012097 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:05.017055 (+ 4958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:05.018602 (+ 1547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:05.018612 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:05.018615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:05.018917 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:05.019551 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:05.019555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:05.019557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:05.020089 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:05.020680 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:05.021147 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 02:46:05.021570 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":999,"thread_start_us":767,"threads_started":1}
I20260812 02:46:05.024127 32396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:05.115849 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 32240
I20260812 02:46:05.140223 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 32408
Aug 12 02:46:05 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:46:05.162528 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:05.141355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.17.196.194:40557)
0812 02:46:05.142189 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:05.142197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:05.142231 (+ 34us) server_negotiation.cc:407] Connection header received
0812 02:46:05.142304 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:05.142313 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:05.142574 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:05.142811 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:05.144933 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:05.146333 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:05.148303 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:05.149006 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:05.149212 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:05.154334 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:05.154351 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:05.154369 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:05.154429 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:05.158724 (+ 4295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:05.159784 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:05.159793 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:05.159795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:05.159964 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:05.160474 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:05.160478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:05.160480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:05.160877 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:05.161160 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:05.161735 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 02:46:05.162015 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":493,"threads_started":1}
I20260812 02:46:05.164204 32564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:05.252971 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 32408
I20260812 02:46:05.277664 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 32577
Aug 12 02:46:05 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:46:05.298677 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:05.278755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.17.196.195:41151)
0812 02:46:05.279548 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:05.279558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:05.279589 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:46:05.279657 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:05.279664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:05.279908 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:05.280156 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:05.281975 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:05.283233 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:05.284710 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:05.285374 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:05.285581 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:05.291272 (+ 5691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:05.291298 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:05.291317 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:05.291373 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:05.294764 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:05.295811 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:05.295820 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:05.295823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:05.296000 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:05.296684 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:05.296689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:05.296691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:05.297138 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:05.297421 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:05.297978 (+ 557us) server_negotiation.cc:299] Negotiation successful
0812 02:46:05.298212 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":425,"threads_started":1}
I20260812 02:46:05.300105 32727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:05.390057 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 32577
I20260812 02:46:05.411525 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 32108
I20260812 02:46:05.413384 32179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:05.529326 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 32108
I20260812 02:46:05.563409 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44729
--webserver_interface=127.17.196.254
--webserver_port=33525
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44729
--ranger_config_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:52991/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.196.212:52991/kms with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:05.925524 32748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:05.925884 32748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:05.925978 32748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:05.933635 32748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:05.933754 32748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:05.933800 32748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:05.933859 32748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:05.933907 32748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:05.944844 32748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.196.212:52991/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44729
--ranger_config_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=33525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:05.946326 32748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:05.947646 32748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:05.960116 32756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:05.960204 32754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:05.960175 32753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:05.961030 32748 server_base.cc:1034] running on GCE node
I20260812 02:46:05.961927 32748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:05.963368 32748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:05.964574 32748 hybrid_clock.cc:648] HybridClock initialized: now 1786502765964525 us; error 43 us; skew 500 ppm
I20260812 02:46:05.964911 32748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:05 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502765, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:05.968389 32748 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:46:05.970129 32748 webserver.cc:460] Webserver started at http://127.17.196.254:33525/ using document root <none> and password file <none>
I20260812 02:46:05.970875 32748 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:05.971042 32748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 02:46:06 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502765, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:06.060703 32748 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:46:06.064113 32766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:06.065892 32748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:46:06.066166 32748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "1bd59b4409a74fdcb542f0e9830e19a4"
format_stamp: "Formatted at 2026-08-12 02:46:01 on dist-test-slave-ss4n"
server_key: "dacd8695fe505ad3d41006848d5dc116"
server_key_iv: "d767cecfe013f55a071155ffeeb89fa8"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:06.067238 32748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:06.108767 32748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:06.113329 32748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:06.113677 32748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:06.134143 32748 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:44729
I20260812 02:46:06.134195 355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:44729 every 8 connection(s)
I20260812 02:46:06.135861 32748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:46:06.136698 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 32748
I20260812 02:46:06.137959 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:52991/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:42739
--local_ip_for_outbound_sockets=127.17.196.193
--tserver_master_addrs=127.17.196.254:44729
--webserver_port=35323
--webserver_interface=127.17.196.193
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.196.212:52991/kms with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:46:06.143450 356 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:46:06.147030 356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Bootstrap starting.
I20260812 02:46:06.174801 356 log.cc:824] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:06.187464 356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:46:06.188330 356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Bootstrap complete.
I20260812 02:46:06.201819 356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } }
I20260812 02:46:06.203203 356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd59b4409a74fdcb542f0e9830e19a4, State: Initialized, Role: FOLLOWER
I20260812 02:46:06.204063 356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } }
I20260812 02:46:06.204370 356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:46:06.204507 356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:46:06.204731 356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:46:06.210821 356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } }
I20260812 02:46:06.211439 356 leader_election.cc:302] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bd59b4409a74fdcb542f0e9830e19a4; no voters:
I20260812 02:46:06.212152 356 leader_election.cc:288] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 02:46:06.212342 361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:46:06.213703 361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [term 2 LEADER]: Becoming Leader. State: Replica: 1bd59b4409a74fdcb542f0e9830e19a4, State: Running, Role: LEADER
I20260812 02:46:06.214288 361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } }
I20260812 02:46:06.214540 356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:06.216821 363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bd59b4409a74fdcb542f0e9830e19a4. Latest consensus state: current_term: 2 leader_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } } }
I20260812 02:46:06.217233 363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:06.218021 362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd59b4409a74fdcb542f0e9830e19a4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44729 } } }
I20260812 02:46:06.218401 362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:06.224018 367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:46:06.229763 367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:46:06.231348 367 catalog_manager.cc:1259] Loaded cluster ID: 6731f524f77f43b5a9a963bb36a1e4aa
I20260812 02:46:06.231501 367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:46:06.235752 367 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:46:06.237564 367 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Loaded TSK: 0
I20260812 02:46:06.239246 367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:46:06.242974 356 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260812 02:46:06.677597 359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:06.678162 359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:06.678391 359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:06.691051 359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:06.691285 359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:06.691560 359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:46:06.708891 359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.196.212:52991/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:42739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=35323
--tserver_master_addrs=127.17.196.254:44729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:06.711086 359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:06.713200 359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:06.732043 403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:06.732014 405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:06.732291 402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:06.732878 359 server_base.cc:1034] running on GCE node
I20260812 02:46:06.733611 359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:06.734854 359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:06.736123 359 hybrid_clock.cc:648] HybridClock initialized: now 1786502766736043 us; error 51 us; skew 500 ppm
I20260812 02:46:06.736519 359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:06 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502766, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:06.746565 359 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:46:06.748698 359 webserver.cc:460] Webserver started at http://127.17.196.193:35323/ using document root <none> and password file <none>
I20260812 02:46:06.749642 359 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:06.749827 359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 02:46:06 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502766, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:06.784338 359 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.013s sys 0.000s
I20260812 02:46:06.791839 416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:06.794203 359 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.003s
I20260812 02:46:06.794466 359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "8983ed99b4bd42d588ca874b0ba7b260"
format_stamp: "Formatted at 2026-08-12 02:46:03 on dist-test-slave-ss4n"
server_key: "5206228402a7a4db33aea8f1bf96ff72"
server_key_iv: "fd7df03f40a74ad03c6772f0b5e2d233"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:06.796046 359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:06.833786 359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:06.840076 359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:06.840520 359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:06.841991 359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:06.846432 359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:06.846554 359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:06.846698 359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:06.846793 359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:06.914403 359 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:42739
I20260812 02:46:06.914482 548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:42739 every 8 connection(s)
I20260812 02:46:06.916540 359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:46:06.926545 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 359
I20260812 02:46:06.927341 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:52991/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:40557
--local_ip_for_outbound_sockets=127.17.196.194
--tserver_master_addrs=127.17.196.254:44729
--webserver_port=34067
--webserver_interface=127.17.196.194
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.196.212:52991/kms with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:46:06 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502766, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:06.947682 555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:06.920009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44729 (local address 127.17.196.193:32781)
0812 02:46:06.921259 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:06.921308 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:06.923402 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:06.925267 (+ 1865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:06.925275 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:06.925943 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:06.926838 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:06.926870 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:06.929254 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:06.929262 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:06.931496 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:06.931518 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:06.933278 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:06.933289 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:06.933676 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:06.934531 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:06.934568 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:06.937830 (+ 3262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:06.942362 (+ 4532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:06.942374 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:06.942393 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:06.943091 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:06.943698 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:06.943703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:06.943708 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:06.943985 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:06.945119 (+ 1134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:06.945128 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:06.945633 (+ 505us) client_negotiation.cc:769] Sending connection context
0812 02:46:06.945955 (+ 322us) client_negotiation.cc:240] Negotiation successful
0812 02:46:06.946591 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"thread_start_us":730,"threads_started":1}
I20260812 02:46:06.950675 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:06.919989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:32781 (local address 127.17.196.254:44729)
0812 02:46:06.920821 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:06.920842 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:06.921509 (+ 667us) server_negotiation.cc:407] Connection header received
0812 02:46:06.924175 (+ 2666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:06.924197 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:06.924715 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:06.925143 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:06.927081 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:06.929036 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:06.931694 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:06.932802 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:06.933039 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:06.938188 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:06.938197 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:06.938214 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:06.938307 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:06.942163 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:06.943287 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:06.943299 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:06.943311 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:06.943550 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:06.944147 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:06.944155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:06.944161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:06.944946 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:06.945175 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:06.948764 (+ 3589us) server_negotiation.cc:299] Negotiation successful
0812 02:46:06.949219 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":319,"threads_started":1}
I20260812 02:46:06.955152 552 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44729
I20260812 02:46:06.955578 552 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:06.956664 552 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
W20260812 02:46:06.960738 552 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:44729 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 02:46:06.961135 552 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:06.961604 552 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
I20260812 02:46:06.963143 552 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:06.963573 552 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
W20260812 02:46:06.964972 552 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:46:07.359804 556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:07.360160 556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:07.360335 556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:07.368436 556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:07.368611 556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:07.368866 556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:46:07.381958 556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.196.212:52991/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:40557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=34067
--tserver_master_addrs=127.17.196.254:44729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:07.383704 556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:07.385576 556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:07.403549 572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:07.403715 571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:07.404939 574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:07.405959 556 server_base.cc:1034] running on GCE node
I20260812 02:46:07.406867 556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:07.408262 556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:07.409564 556 hybrid_clock.cc:648] HybridClock initialized: now 1786502767409476 us; error 67 us; skew 500 ppm
I20260812 02:46:07.409968 556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:07 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:07.414356 556 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:46:07.416872 556 webserver.cc:460] Webserver started at http://127.17.196.194:34067/ using document root <none> and password file <none>
I20260812 02:46:07.417937 556 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:07.418128 556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 02:46:07 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:07.454792 556 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.004s sys 0.008s
I20260812 02:46:07.460858 583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:07.462719 556 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.002s
I20260812 02:46:07.462947 556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "a4e38fafa85b47a79b851383e14ede87"
format_stamp: "Formatted at 2026-08-12 02:46:04 on dist-test-slave-ss4n"
server_key: "266d33030161313bf6df933dfffaab56"
server_key_iv: "981bed87723f21981b4ac9f7fdff172c"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:07.464061 556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:07.513729 556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:07.519939 556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:07.520309 556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:07.524986 556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:07.526958 556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:07.527112 556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:07.527232 556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:07.527330 556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:07.578758 556 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:40557
I20260812 02:46:07.578847 717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:40557 every 8 connection(s)
I20260812 02:46:07.580791 556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:46:07.584326 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 556
I20260812 02:46:07.585374 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.196.212:52991/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:41151
--local_ip_for_outbound_sockets=127.17.196.195
--tserver_master_addrs=127.17.196.254:44729
--webserver_port=46827
--webserver_interface=127.17.196.195
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.196.212:52991/kms with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:46:07 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 02:46:07 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:07.644636 723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:07.585604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:45697 (local address 127.17.196.254:44729)
0812 02:46:07.606939 (+ 21335us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:07.606947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:07.611205 (+ 4258us) server_negotiation.cc:407] Connection header received
0812 02:46:07.611295 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:07.611301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:07.611423 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:07.611533 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:07.618620 (+ 7087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:07.619847 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:07.629515 (+ 9668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:07.630573 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:07.630757 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:07.636295 (+ 5538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:07.636305 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:07.636312 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:07.636366 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:07.640575 (+ 4209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:07.641676 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:07.641682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:07.641686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:07.641847 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:07.642413 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:07.642418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:07.642422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:07.642795 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:07.642925 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:07.643967 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0812 02:46:07.644216 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21168,"thread_start_us":21349,"threads_started":1}
I20260812 02:46:07.647392 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:07.584860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44729 (local address 127.17.196.194:45697)
0812 02:46:07.607753 (+ 22893us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:07.607822 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:07.610659 (+ 2837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:07.613542 (+ 2883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:07.613558 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:07.616185 (+ 2627us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:07.617915 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:07.617947 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:07.626689 (+ 8742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:07.626697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:07.629201 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:07.629229 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:07.631200 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:07.631213 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:07.631631 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:07.632586 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:07.632639 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:07.635926 (+ 3287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:07.640797 (+ 4871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:07.640809 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:07.640830 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:07.641506 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:07.641995 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:07.642000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:07.642003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:07.642279 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:07.643037 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:07.643047 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:07.643507 (+ 460us) client_negotiation.cc:769] Sending connection context
0812 02:46:07.645192 (+ 1685us) client_negotiation.cc:240] Negotiation successful
0812 02:46:07.645856 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22499,"thread_start_us":26672,"threads_started":1}
I20260812 02:46:07.649586 719 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44729
I20260812 02:46:07.649921 719 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:07.650792 719 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
I20260812 02:46:07.653002 317 ts_manager.cc:194] Registered new tserver with Master: a4e38fafa85b47a79b851383e14ede87 (127.17.196.194:40557)
I20260812 02:46:07.655220 317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:45697
I20260812 02:46:07.965159 552 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:07.965689 552 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
I20260812 02:46:07.967073 317 ts_manager.cc:194] Registered new tserver with Master: 8983ed99b4bd42d588ca874b0ba7b260 (127.17.196.193:42739)
I20260812 02:46:07.968240 317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:32781
W20260812 02:46:08.066673 724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:08.067049 724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:08.067243 724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:08.076511 724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:08.076694 724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:08.076972 724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:46:08.091183 724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.196.212:52991/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:41151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=46827
--tserver_master_addrs=127.17.196.254:44729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:08.093194 724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:08.095114 724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:08.120258 731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:08.125741 732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:08.128149 724 server_base.cc:1034] running on GCE node
W20260812 02:46:08.128942 734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:08.129957 724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
Aug 12 02:46:08 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:08.136807 724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:08.138230 724 hybrid_clock.cc:648] HybridClock initialized: now 1786502768138103 us; error 114 us; skew 500 ppm
I20260812 02:46:08.138706 724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:08 dist-test-slave-ss4n krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:08.143821 724 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:46:08.149938 724 webserver.cc:460] Webserver started at http://127.17.196.195:46827/ using document root <none> and password file <none>
I20260812 02:46:08.151124 724 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:08.151319 724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 02:46:08 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:08.214672 724 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.007s sys 0.003s
I20260812 02:46:08.230307 742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:08.232657 724 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.006s sys 0.009s
I20260812 02:46:08.233001 724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "0e45c2f653fd4713bf5a2b18357f64be"
format_stamp: "Formatted at 2026-08-12 02:46:04 on dist-test-slave-ss4n"
server_key: "9e059bf67ff68d86cb6c10f515def4c1"
server_key_iv: "61f6b71e7d4b887e761bf23dac8fba70"
server_key_version: "kuduclusterkey@0"
I20260812 02:46:08.234769 724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:08.281804 724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:08.288231 724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:08.288677 724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:08.302191 724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:08.306588 724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:08.306751 724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:08.306962 724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:08.307057 724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:08.356890 724 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:41151
I20260812 02:46:08.356937 871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:41151 every 8 connection(s)
I20260812 02:46:08.358721 724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:46:08.365900 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 724
Aug 12 02:46:08 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:08.393043 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.362839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44729 (local address 127.17.196.195:60513)
0812 02:46:08.363709 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:08.363764 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:08.365814 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:08.367029 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:08.367043 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:08.367831 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:08.368909 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.368942 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.371717 (+ 2775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.371727 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.374635 (+ 2908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.374681 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.376610 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.376620 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.377071 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.378141 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:08.378186 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:08.382064 (+ 3878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:08.387392 (+ 5328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:08.387406 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:08.387432 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:08.388224 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:08.388731 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:08.388736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:08.388740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:08.388993 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:08.389917 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:08.389926 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:08.390391 (+ 465us) client_negotiation.cc:769] Sending connection context
0812 02:46:08.390745 (+ 354us) client_negotiation.cc:240] Negotiation successful
0812 02:46:08.391369 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":371,"threads_started":1}
I20260812 02:46:08.394387 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.362261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:60513 (local address 127.17.196.254:44729)
0812 02:46:08.362893 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.362900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.363876 (+ 976us) server_negotiation.cc:407] Connection header received
0812 02:46:08.366220 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.366227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.366329 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.366419 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:08.369216 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.370816 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.374919 (+ 4103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.375936 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.376089 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.382511 (+ 6422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:08.382522 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:08.382529 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:08.382610 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:08.387184 (+ 4574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:08.388383 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:08.388388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:08.388392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:08.388588 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:08.389131 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:08.389135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:08.389138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:08.389660 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:08.389825 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.393651 (+ 3826us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.393916 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":352,"threads_started":1}
I20260812 02:46:08.396953 872 heartbeater.cc:347] Connected to a master server at 127.17.196.254:44729
I20260812 02:46:08.397325 872 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:08.398422 872 heartbeater.cc:511] Master 127.17.196.254:44729 requested a full tablet report, sending...
I20260812 02:46:08.400403 317 ts_manager.cc:194] Registered new tserver with Master: 0e45c2f653fd4713bf5a2b18357f64be (127.17.196.195:41151)
I20260812 02:46:08.401670 317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:60513
I20260812 02:46:08.404106 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.367053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.17.196.254:44729)
0812 02:46:08.384950 (+ 17897us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.384955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.384971 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:46:08.385030 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.385033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.385142 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.385255 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:08.387741 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.388944 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.391003 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.391643 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.391763 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.395760 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:08.395771 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:08.395775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:08.395814 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:08.399646 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:08.400717 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:08.400723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:08.400727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:08.400879 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:08.401373 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:08.401379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:08.401398 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:08.401856 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:08.402193 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.403458 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.403718 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17770,"thread_start_us":17606,"threads_started":1}
I20260812 02:46:08.407481 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:46:08.429389 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.411020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.17.196.254:44729)
0812 02:46:08.411232 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.411241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.414116 (+ 2875us) server_negotiation.cc:407] Connection header received
0812 02:46:08.414344 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.414350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.414427 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.414536 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:08.416170 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.416887 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.418407 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.418869 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.419022 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.422041 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:08.422048 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:08.422052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:08.422097 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:08.425921 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:08.426902 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:08.426908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:08.426911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:08.427079 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:08.427618 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:08.427623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:08.427626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:08.427978 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:08.428177 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.428723 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.428974 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:46:08.434945 317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:46:08.443485 317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:46:08.494200 891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.460016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:41151 (local address 127.0.0.1:50724)
0812 02:46:08.461423 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:08.461472 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:08.464466 (+ 2994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:08.465845 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:08.465853 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:08.465884 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:08.467342 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.467361 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.473740 (+ 6379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.473746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.481093 (+ 7347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.481115 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.492591 (+ 11476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.492602 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.492999 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.493121 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 02:46:08.493301 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 02:46:08.493603 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"mutex_wait_us":29,"thread_start_us":994,"threads_started":1}
I20260812 02:46:08.496356 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.460763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.17.196.195:41151)
0812 02:46:08.465115 (+ 4352us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.465125 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.465157 (+ 32us) server_negotiation.cc:407] Connection header received
0812 02:46:08.465266 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.465274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.465592 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.467569 (+ 1977us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:08.467868 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.473554 (+ 5686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.482544 (+ 8990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.486646 (+ 4102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.492969 (+ 6323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.493217 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.495576 (+ 2359us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.495866 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4176,"thread_start_us":8798,"threads_started":1}
I20260812 02:46:08.501540 797 tablet_service.cc:1467] Processing CreateTablet for tablet ae85efcf68374cad9ee110ea7fa738f1 (DEFAULT_TABLE table=test-table [id=8e325df007d54d5aa6f9ac4bae4af56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:08.503461 797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae85efcf68374cad9ee110ea7fa738f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:08.513270 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.484308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39490 (local address 127.17.196.193:42739)
0812 02:46:08.502099 (+ 17791us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.502108 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.502163 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:46:08.502272 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.502280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.502559 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.502774 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:08.504884 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.506435 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.509041 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.510435 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.510651 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.510831 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.512776 (+ 1945us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.512958 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17602,"thread_start_us":19178,"threads_started":1}
I20260812 02:46:08.523708 893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.464790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:40557 (local address 127.0.0.1:40104)
0812 02:46:08.483686 (+ 18896us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:08.483710 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:08.483838 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:08.487107 (+ 3269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:08.487115 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:08.487162 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:08.487705 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.487724 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.490450 (+ 2726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.490459 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.516454 (+ 25995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.516476 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.522715 (+ 6239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.522724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.522942 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.522985 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 02:46:08.523101 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 02:46:08.523273 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18743,"mutex_wait_us":5,"thread_start_us":18588,"threads_started":1}
I20260812 02:46:08.524150 891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.491163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:42739 (local address 127.0.0.1:39490)
0812 02:46:08.495152 (+ 3989us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:08.495176 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:08.495299 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:08.504459 (+ 9160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:08.504464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:08.504496 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:08.504676 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.504687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.506624 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.506629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.508323 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.508349 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.511716 (+ 3367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.511723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.511918 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.511952 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:46:08.512047 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 02:46:08.512198 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3826,"thread_start_us":12119,"threads_started":1}
I20260812 02:46:08.528954 471 tablet_service.cc:1467] Processing CreateTablet for tablet ae85efcf68374cad9ee110ea7fa738f1 (DEFAULT_TABLE table=test-table [id=8e325df007d54d5aa6f9ac4bae4af56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:08.530668 471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae85efcf68374cad9ee110ea7fa738f1. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:46:08 dist-test-slave-ss4n krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for HTTP/127.17.196.212@KRBTEST.COM
I20260812 02:46:08.534301 900 tablet_bootstrap.cc:492] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Bootstrap starting.
I20260812 02:46:08.538506 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.465391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.17.196.194:40557)
0812 02:46:08.484408 (+ 19017us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.484418 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.484454 (+ 36us) server_negotiation.cc:407] Connection header received
0812 02:46:08.484587 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.484599 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.484979 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.486699 (+ 1720us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:08.487995 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.490225 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.516832 (+ 26607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.518379 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.518599 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.518780 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.537863 (+ 19083us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.538127 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18830,"thread_start_us":24714,"threads_started":1}
I20260812 02:46:08.542965 637 tablet_service.cc:1467] Processing CreateTablet for tablet ae85efcf68374cad9ee110ea7fa738f1 (DEFAULT_TABLE table=test-table [id=8e325df007d54d5aa6f9ac4bae4af56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:08.544582 637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae85efcf68374cad9ee110ea7fa738f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:08.546972 900 tablet_bootstrap.cc:654] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:08.554967 900 log.cc:824] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:08.565861 900 tablet_bootstrap.cc:492] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: No bootstrap required, opened a new log
I20260812 02:46:08.566469 900 ts_tablet_manager.cc:1397] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.000s
I20260812 02:46:08.568310 903 tablet_bootstrap.cc:492] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Bootstrap starting.
I20260812 02:46:08.574404 901 tablet_bootstrap.cc:492] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Bootstrap starting.
I20260812 02:46:08.575608 903 tablet_bootstrap.cc:654] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:08.577344 903 log.cc:824] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:08.578944 901 tablet_bootstrap.cc:654] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:08.580758 901 log.cc:824] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:08.587796 903 tablet_bootstrap.cc:492] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: No bootstrap required, opened a new log
I20260812 02:46:08.587786 901 tablet_bootstrap.cc:492] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: No bootstrap required, opened a new log
I20260812 02:46:08.588187 903 ts_tablet_manager.cc:1397] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.007s
I20260812 02:46:08.588207 901 ts_tablet_manager.cc:1397] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260812 02:46:08.588936 900 raft_consensus.cc:348] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.589437 900 raft_consensus.cc:374] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:08.589567 900 raft_consensus.cc:729] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e45c2f653fd4713bf5a2b18357f64be, State: Initialized, Role: FOLLOWER
I20260812 02:46:08.596112 900 consensus_queue.cc:260] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.597663 872 heartbeater.cc:503] Master 127.17.196.254:44729 was elected leader, sending a full tablet report...
I20260812 02:46:08.601713 901 raft_consensus.cc:348] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.602229 903 raft_consensus.cc:348] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.602524 901 raft_consensus.cc:374] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:08.602721 903 raft_consensus.cc:374] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:08.602895 901 raft_consensus.cc:729] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8983ed99b4bd42d588ca874b0ba7b260, State: Initialized, Role: FOLLOWER
I20260812 02:46:08.602902 903 raft_consensus.cc:729] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e38fafa85b47a79b851383e14ede87, State: Initialized, Role: FOLLOWER
I20260812 02:46:08.603809 901 consensus_queue.cc:260] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.603809 903 consensus_queue.cc:260] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.606946 900 ts_tablet_manager.cc:1428] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Time spent starting tablet: real 0.040s user 0.014s sys 0.000s
W20260812 02:46:08.612267 873 tablet.cc:2369] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:08.612793 903 ts_tablet_manager.cc:1428] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Time spent starting tablet: real 0.024s user 0.015s sys 0.009s
I20260812 02:46:08.613824 552 heartbeater.cc:503] Master 127.17.196.254:44729 was elected leader, sending a full tablet report...
W20260812 02:46:08.614530 720 tablet.cc:2369] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:08.614740 901 ts_tablet_manager.cc:1428] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Time spent starting tablet: real 0.026s user 0.015s sys 0.010s
I20260812 02:46:08.615365 719 heartbeater.cc:503] Master 127.17.196.254:44729 was elected leader, sending a full tablet report...
W20260812 02:46:08.669826 553 tablet.cc:2369] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:08.826607 907 raft_consensus.cc:482] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:46:08.826995 907 raft_consensus.cc:504] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.829655 907 leader_election.cc:288] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e45c2f653fd4713bf5a2b18357f64be (127.17.196.195:41151), 8983ed99b4bd42d588ca874b0ba7b260 (127.17.196.193:42739)
I20260812 02:46:08.842795 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.830689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:37117 (local address 127.17.196.193:42739)
0812 02:46:08.831004 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.831010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.832964 (+ 1954us) server_negotiation.cc:407] Connection header received
0812 02:46:08.833355 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.833362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.833443 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.833611 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:08.834869 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.836068 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.839438 (+ 3370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.841135 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.841365 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.841519 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.842124 (+ 605us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.842309 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:46:08.843175 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.830010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:42739 (local address 127.17.196.194:37117)
0812 02:46:08.832829 (+ 2819us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:08.832848 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:08.833165 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:08.833623 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:08.833628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:08.833655 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:08.834688 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.834705 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.836227 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.836235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.839218 (+ 2983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.839232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.841450 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.841458 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.841865 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.841895 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 02:46:08.842082 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 02:46:08.842313 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2574,"thread_start_us":2122,"threads_started":1}
I20260812 02:46:08.844630 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.830670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:43479 (local address 127.17.196.195:41151)
0812 02:46:08.830944 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.830953 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.833153 (+ 2200us) server_negotiation.cc:407] Connection header received
0812 02:46:08.833476 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.833506 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.833582 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.833646 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:08.835433 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.837077 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.840855 (+ 3778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.842719 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.842905 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.843008 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.843967 (+ 959us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.844216 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 02:46:08.844887 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.830055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:41151 (local address 127.17.196.194:43479)
0812 02:46:08.833033 (+ 2978us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:08.833051 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:08.833300 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:08.833887 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:08.833892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:08.833929 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:08.835031 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.835046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.837296 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.837301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.840611 (+ 3310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:08.840627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.843404 (+ 2777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.843413 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:08.843607 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.843652 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 02:46:08.843769 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 02:46:08.844177 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2723,"thread_start_us":1427,"threads_started":1}
I20260812 02:46:08.845929 495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae85efcf68374cad9ee110ea7fa738f1" candidate_uuid: "a4e38fafa85b47a79b851383e14ede87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8983ed99b4bd42d588ca874b0ba7b260" is_pre_election: true
I20260812 02:46:08.846964 820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae85efcf68374cad9ee110ea7fa738f1" candidate_uuid: "a4e38fafa85b47a79b851383e14ede87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e45c2f653fd4713bf5a2b18357f64be" is_pre_election: true
I20260812 02:46:08.847324 495 raft_consensus.cc:2454] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e38fafa85b47a79b851383e14ede87 in term 0.
I20260812 02:46:08.847993 820 raft_consensus.cc:2454] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e38fafa85b47a79b851383e14ede87 in term 0.
I20260812 02:46:08.848608 584 leader_election.cc:302] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8983ed99b4bd42d588ca874b0ba7b260, a4e38fafa85b47a79b851383e14ede87; no voters:
I20260812 02:46:08.849251 907 raft_consensus.cc:2781] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:46:08.849457 907 raft_consensus.cc:482] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:46:08.849731 907 raft_consensus.cc:3037] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:08.854452 907 raft_consensus.cc:504] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.855381 907 leader_election.cc:288] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [CANDIDATE]: Term 1 election: Requested vote from peers 0e45c2f653fd4713bf5a2b18357f64be (127.17.196.195:41151), 8983ed99b4bd42d588ca874b0ba7b260 (127.17.196.193:42739)
I20260812 02:46:08.856236 495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae85efcf68374cad9ee110ea7fa738f1" candidate_uuid: "a4e38fafa85b47a79b851383e14ede87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8983ed99b4bd42d588ca874b0ba7b260"
I20260812 02:46:08.856236 820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae85efcf68374cad9ee110ea7fa738f1" candidate_uuid: "a4e38fafa85b47a79b851383e14ede87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e45c2f653fd4713bf5a2b18357f64be"
I20260812 02:46:08.856551 495 raft_consensus.cc:3037] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:08.856550 820 raft_consensus.cc:3037] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:08.860240 495 raft_consensus.cc:2454] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e38fafa85b47a79b851383e14ede87 in term 1.
I20260812 02:46:08.860960 584 leader_election.cc:302] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8983ed99b4bd42d588ca874b0ba7b260, a4e38fafa85b47a79b851383e14ede87; no voters:
I20260812 02:46:08.861465 907 raft_consensus.cc:2781] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:08.861734 820 raft_consensus.cc:2454] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e38fafa85b47a79b851383e14ede87 in term 1.
I20260812 02:46:08.862752 907 raft_consensus.cc:686] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 1 LEADER]: Becoming Leader. State: Replica: a4e38fafa85b47a79b851383e14ede87, State: Running, Role: LEADER
I20260812 02:46:08.863722 907 consensus_queue.cc:237] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } }
I20260812 02:46:08.870838 317 catalog_manager.cc:5495] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4e38fafa85b47a79b851383e14ede87 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "a4e38fafa85b47a79b851383e14ede87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4e38fafa85b47a79b851383e14ede87" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 } health_report { overall_health: UNKNOWN } } }
I20260812 02:46:08.935781 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:08.927151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.17.196.194:40557)
0812 02:46:08.927459 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:08.927467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:08.927493 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:46:08.927808 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:08.927812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:08.927951 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:08.928144 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:46:08.928693 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.930174 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.932360 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:08.933059 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:08.933245 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:08.934338 (+ 1093us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:46:08.934942 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:46:08.935136 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:08.935251 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 02:46:08.935351 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 02:46:08.948807 495 raft_consensus.cc:1261] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 1 FOLLOWER]: Refusing update from remote peer a4e38fafa85b47a79b851383e14ede87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:08.948879 820 raft_consensus.cc:1261] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 1 FOLLOWER]: Refusing update from remote peer a4e38fafa85b47a79b851383e14ede87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:08.950366 920 consensus_queue.cc:1035] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e45c2f653fd4713bf5a2b18357f64be" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 02:46:08.951170 907 consensus_queue.cc:1035] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8983ed99b4bd42d588ca874b0ba7b260" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 42739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:08.967952 923 mvcc.cc:205] Tried to move back new op lower bound from 7317515341599387648 to 7317515341270876160. Current Snapshot: MvccSnapshot[applied={T|T < 7317515341599387648}]
I20260812 02:46:09.005998 316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:46:09.006435 316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:46:09.012322 316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bd59b4409a74fdcb542f0e9830e19a4: Sending DeleteTablet for 3 replicas of tablet ae85efcf68374cad9ee110ea7fa738f1
I20260812 02:46:09.014072 797 tablet_service.cc:1514] Processing DeleteTablet for tablet ae85efcf68374cad9ee110ea7fa738f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:09 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:50724
I20260812 02:46:09.015069 471 tablet_service.cc:1514] Processing DeleteTablet for tablet ae85efcf68374cad9ee110ea7fa738f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:09 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:39490
I20260812 02:46:09.015724 637 tablet_service.cc:1514] Processing DeleteTablet for tablet ae85efcf68374cad9ee110ea7fa738f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:09 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:40104
I20260812 02:46:09.019537 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 359
I20260812 02:46:09.035581 935 tablet_replica.cc:330] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: stopping tablet replica
I20260812 02:46:09.037920 935 raft_consensus.cc:2229] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:46:09.041574 934 tablet_replica.cc:330] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: stopping tablet replica
I20260812 02:46:09.042277 935 raft_consensus.cc:2258] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:09.042500 934 raft_consensus.cc:2229] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:46:09.043251 934 raft_consensus.cc:2258] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:09.046964 933 tablet_replica.cc:330] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: stopping tablet replica
I20260812 02:46:09.047387 935 ts_tablet_manager.cc:1905] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:09.048005 934 ts_tablet_manager.cc:1905] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:09.049883 933 raft_consensus.cc:2229] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:46:09.057467 933 raft_consensus.cc:2258] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:09.060153 934 ts_tablet_manager.cc:1918] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:09.062793 933 ts_tablet_manager.cc:1905] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:09.063123 934 log.cc:1196] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/ae85efcf68374cad9ee110ea7fa738f1
I20260812 02:46:09.064148 934 ts_tablet_manager.cc:1939] T ae85efcf68374cad9ee110ea7fa738f1 P 8983ed99b4bd42d588ca874b0ba7b260: Deleting consensus metadata
I20260812 02:46:09.071070 935 ts_tablet_manager.cc:1918] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:09.071573 935 log.cc:1196] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/ae85efcf68374cad9ee110ea7fa738f1
I20260812 02:46:09.072774 935 ts_tablet_manager.cc:1939] T ae85efcf68374cad9ee110ea7fa738f1 P a4e38fafa85b47a79b851383e14ede87: Deleting consensus metadata
I20260812 02:46:09.073982 300 catalog_manager.cc:4841] TS 8983ed99b4bd42d588ca874b0ba7b260 (127.17.196.193:42739): tablet ae85efcf68374cad9ee110ea7fa738f1 (table test-table [id=8e325df007d54d5aa6f9ac4bae4af56c]) successfully deleted
I20260812 02:46:09.076138 300 catalog_manager.cc:4841] TS a4e38fafa85b47a79b851383e14ede87 (127.17.196.194:40557): tablet ae85efcf68374cad9ee110ea7fa738f1 (table test-table [id=8e325df007d54d5aa6f9ac4bae4af56c]) successfully deleted
I20260812 02:46:09.079604 933 ts_tablet_manager.cc:1918] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:09.080063 933 log.cc:1196] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/ae85efcf68374cad9ee110ea7fa738f1
I20260812 02:46:09.081120 933 ts_tablet_manager.cc:1939] T ae85efcf68374cad9ee110ea7fa738f1 P 0e45c2f653fd4713bf5a2b18357f64be: Deleting consensus metadata
I20260812 02:46:09.082118 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:09.021185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.17.196.193:42739)
0812 02:46:09.021418 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:09.021428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:09.045319 (+ 23891us) server_negotiation.cc:407] Connection header received
0812 02:46:09.045434 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:09.045440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:09.045561 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:09.045691 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:09.048569 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:09.050393 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:09.059388 (+ 8995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:09.060518 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:09.060669 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:09.067346 (+ 6677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:09.067367 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:09.067392 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:09.067475 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:09.075192 (+ 7717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:09.076776 (+ 1584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:09.076802 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:09.076806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:09.077124 (+ 318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:09.077875 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:09.077881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:09.077885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:09.078570 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:09.079212 (+ 642us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:09.080932 (+ 1720us) server_negotiation.cc:299] Negotiation successful
0812 02:46:09.081334 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:46:09.083467 543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:09.083866 301 catalog_manager.cc:4841] TS 0e45c2f653fd4713bf5a2b18357f64be (127.17.196.195:41151): tablet ae85efcf68374cad9ee110ea7fa738f1 (table test-table [id=8e325df007d54d5aa6f9ac4bae4af56c]) successfully deleted
I20260812 02:46:09.203657 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 359
I20260812 02:46:09.229557 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 556
I20260812 02:46:09.248572 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:09.230645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.17.196.194:40557)
0812 02:46:09.230874 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:09.230883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:09.230925 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:46:09.231005 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:09.231012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:09.231098 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:09.231219 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:09.232681 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:09.234701 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:09.236125 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:09.237113 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:09.237261 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:09.240343 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:09.240367 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:09.240393 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:09.240547 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:09.244745 (+ 4198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:09.246055 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:09.246064 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:09.246066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:09.246247 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:09.246825 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:09.246830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:09.246832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:09.247295 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:09.247635 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:09.248039 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 02:46:09.248231 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:46:09.249303 713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:09.366577 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 556
I20260812 02:46:09.386327 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 724
I20260812 02:46:09.401168 939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:09.387124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50730 (local address 127.17.196.195:41151)
0812 02:46:09.387727 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:09.387731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:09.387747 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:46:09.387802 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:09.387805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:09.387877 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:09.388009 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:09.389426 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:09.390399 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:09.391829 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:09.392259 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:09.392396 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:09.395099 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:09.395115 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:09.395131 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:09.395171 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:09.398151 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:09.398938 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:09.398946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:09.398948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:09.399087 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:09.399477 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:09.399483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:09.399486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:09.399872 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:09.400187 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:09.400521 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 02:46:09.400772 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":271,"threads_started":1}
I20260812 02:46:09.402101 864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:09.496510 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 724
I20260812 02:46:09.519426 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 32748
I20260812 02:46:09.520771 351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:09.636205 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 32748
I20260812 02:46:09.664186 18195 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 02:46:10.010715 18195 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 02:46:10.010962 18195 mini_ranger.cc:66] Stopping Ranger...
I20260812 02:46:10.357867 18195 mini_ranger.cc:68] Stopped Ranger
2026-08-12 02:46:10.358 UTC [31621] LOG: received smart shutdown request
2026-08-12 02:46:10.362 UTC [31621] LOG: background worker "logical replication launcher" (PID 31631) exited with exit code 1
2026-08-12 02:46:10.362 UTC [31626] LOG: shutting down
2026-08-12 02:46:10.583 UTC [31621] LOG: database system is shut down
2026-08-12T02:46:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (57354 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:46:10 dist-test-slave-ss4n krb5kdc[950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:46:10 dist-test-slave-ss4n krb5kdc[950](info): set up 2 sockets
Aug 12 02:46:10 dist-test-slave-ss4n krb5kdc[950](info): commencing operation
krb5kdc: starting...
W20260812 02:46:11.990031 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:46:12 dist-test-slave-ss4n krb5kdc[950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:46:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:46:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:46:12.298478 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44529
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44529,127.17.196.253:37263,127.17.196.252:40057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:12.657646 966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:12.657982 966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:12.658079 966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:12.665823 966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:12.665962 966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:12.666009 966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:12.666082 966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:12.666136 966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:12.676641 966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44529,127.17.196.253:37263,127.17.196.252:40057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:12.678248 966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:12.679692 966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:12.693075 972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:12.693158 975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:12.693029 973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:12.694530 966 server_base.cc:1034] running on GCE node
I20260812 02:46:12.695369 966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:12.697021 966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:12.698297 966 hybrid_clock.cc:648] HybridClock initialized: now 1786502772698237 us; error 41 us; skew 500 ppm
I20260812 02:46:12.698824 966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:12 dist-test-slave-ss4n krb5kdc[950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502772, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:12.706956 966 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:46:12.708896 966 webserver.cc:460] Webserver started at http://127.17.196.254:34181/ using document root <none> and password file <none>
I20260812 02:46:12.709854 966 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:12.710029 966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:12.710412 966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:12.714042 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "0f237d20ad4744e881703fedd7b26754"
format_stamp: "Formatted at 2026-08-12 02:46:12 on dist-test-slave-ss4n"
server_key: "4fc92a2c4ba5606d3b0aaec9cee8d4ff"
server_key_iv: "d4a72fd5b8063f7ced6d176bada9e475"
server_key_version: "encryptionkey@0"
I20260812 02:46:12.714989 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "0f237d20ad4744e881703fedd7b26754"
format_stamp: "Formatted at 2026-08-12 02:46:12 on dist-test-slave-ss4n"
server_key: "4fc92a2c4ba5606d3b0aaec9cee8d4ff"
server_key_iv: "d4a72fd5b8063f7ced6d176bada9e475"
server_key_version: "encryptionkey@0"
I20260812 02:46:12.721266 966 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 02:46:12.725355 983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:12.727136 966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:46:12.727370 966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "0f237d20ad4744e881703fedd7b26754"
format_stamp: "Formatted at 2026-08-12 02:46:12 on dist-test-slave-ss4n"
server_key: "4fc92a2c4ba5606d3b0aaec9cee8d4ff"
server_key_iv: "d4a72fd5b8063f7ced6d176bada9e475"
server_key_version: "encryptionkey@0"
I20260812 02:46:12.727623 966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:12.758654 966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:12.763984 966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:12.764339 966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:12.786180 966 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:44529
I20260812 02:46:12.786217 1049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:44529 every 8 connection(s)
I20260812 02:46:12.787782 966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:46:12.789261 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 966
I20260812 02:46:12.789578 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:46:12.790526 18195 external_mini_cluster.cc:1428] Setting key 65e30006618f4a47112084e3e4c2fed5
I20260812 02:46:12.793642 1050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:12.805361 1050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.196.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.253@KRBTEST.COM" created.
Aug 12 02:46:12 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502772, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:12.834100 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:12.809307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44529 (local address 127.0.0.1:42158)
0812 02:46:12.810630 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:12.810673 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:12.812522 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:12.813411 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:12.813418 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:12.814198 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:12.814559 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:12.814577 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:12.816213 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:12.816219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:12.817318 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:12.817331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:12.818363 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:12.818370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:12.818556 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:12.819243 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:12.819268 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:12.824211 (+ 4943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:12.829096 (+ 4885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:12.829107 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:12.829117 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:12.829624 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:12.830167 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:12.830171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:12.830174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:12.830367 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:12.831563 (+ 1196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:12.831573 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:12.831824 (+ 251us) client_negotiation.cc:769] Sending connection context
0812 02:46:12.832146 (+ 322us) client_negotiation.cc:240] Negotiation successful
0812 02:46:12.832939 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"mutex_wait_us":237,"spinlock_wait_cycles":29056,"thread_start_us":883,"threads_started":1}
I20260812 02:46:12.834157 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:12.809293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42158 (local address 127.17.196.254:44529)
0812 02:46:12.810287 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:12.810307 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:12.810805 (+ 498us) server_negotiation.cc:407] Connection header received
0812 02:46:12.812933 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:12.812941 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:12.813172 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:12.813385 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:12.814760 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:12.815975 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:12.817529 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:12.818118 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:12.818322 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:12.824500 (+ 6178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:12.824515 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:12.824536 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:12.824583 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:12.828895 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:12.829778 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:12.829790 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:12.829796 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:12.830004 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:12.830504 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:12.830511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:12.830517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:12.831358 (+ 841us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:12.831579 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:12.832361 (+ 782us) server_negotiation.cc:299] Negotiation successful
0812 02:46:12.832796 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":747,"thread_start_us":504,"threads_started":1}
I20260812 02:46:12.837749 1050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:46:12.839018 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:12.838404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47608)
0812 02:46:12.838640 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:12.838825 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 02:46:12.839294 985 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
I20260812 02:46:12.841101 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:12.840580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47624)
0812 02:46:12.840787 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:12.840893 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 02:46:12.841359 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:37263: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab.
Entry for principal kudu/127.17.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab.
I20260812 02:46:12.861445 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.253:37263
--webserver_interface=127.17.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab
--principal=kudu/127.17.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44529,127.17.196.253:37263,127.17.196.252:40057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:46:12.890906 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } attempt: 1
I20260812 02:46:12.892264 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:12.891724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47638)
0812 02:46:12.891950 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:12.892117 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 02:46:12.893589 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:12.893095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47648)
0812 02:46:12.893296 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:12.893378 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 02:46:12.893826 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:37263: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
I20260812 02:46:12.962129 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } attempt: 2
I20260812 02:46:12.963464 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:12.962941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47656)
0812 02:46:12.963215 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:12.963319 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 02:46:12.964795 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:12.964247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47660)
0812 02:46:12.964441 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:12.964553 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 02:46:12.965085 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:37263: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
I20260812 02:46:13.056437 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } attempt: 3
I20260812 02:46:13.057866 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.057259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47668)
0812 02:46:13.057609 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.057711 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260812 02:46:13.059095 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.058717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47678)
0812 02:46:13.058863 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.058951 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 02:46:13.059376 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:37263: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
I20260812 02:46:13.202646 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } attempt: 4
I20260812 02:46:13.204196 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.203490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47686)
0812 02:46:13.203826 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.204038 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200}
I20260812 02:46:13.205415 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.204975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47696)
0812 02:46:13.205170 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.205265 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 02:46:13.205725 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:37263: Network error: Client connection negotiation failed: client connection to 127.17.196.253:37263: connect: Connection refused (error 111)
W20260812 02:46:13.219187 1057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:13.219591 1057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:13.219754 1057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:13.227790 1057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:13.227977 1057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:13.228099 1057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:13.228209 1057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:13.228288 1057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:13.238951 1057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44529,127.17.196.253:37263,127.17.196.252:40057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab
--principal=kudu/127.17.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.253:37263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:13.240476 1057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:13.242017 1057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:13.254698 1062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:13.254681 1066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:13.254854 1063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:13.255549 1057 server_base.cc:1034] running on GCE node
I20260812 02:46:13.256405 1057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:13.257972 1057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:13.259234 1057 hybrid_clock.cc:648] HybridClock initialized: now 1786502773259185 us; error 40 us; skew 500 ppm
I20260812 02:46:13.259624 1057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:13.263383 1057 init.cc:377] Logged in from keytab as kudu/127.17.196.253@KRBTEST.COM (short username kudu)
I20260812 02:46:13.265026 1057 webserver.cc:460] Webserver started at http://127.17.196.253:42387/ using document root <none> and password file <none>
I20260812 02:46:13.265851 1057 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:13.265975 1057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:13.266258 1057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:13.269673 1057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/data/instance:
uuid: "59d3f56707af4830a4eda6832b836d89"
format_stamp: "Formatted at 2026-08-12 02:46:13 on dist-test-slave-ss4n"
server_key: "344b386c31b88fef49bd386d85d5aa72"
server_key_iv: "c43f14f0e906fa2a97fb77ae3e66b409"
server_key_version: "encryptionkey@0"
I20260812 02:46:13.270536 1057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/wal/instance:
uuid: "59d3f56707af4830a4eda6832b836d89"
format_stamp: "Formatted at 2026-08-12 02:46:13 on dist-test-slave-ss4n"
server_key: "344b386c31b88fef49bd386d85d5aa72"
server_key_iv: "c43f14f0e906fa2a97fb77ae3e66b409"
server_key_version: "encryptionkey@0"
I20260812 02:46:13.276435 1057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:46:13.280756 1074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:13.282519 1057 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:46:13.282719 1057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/wal
uuid: "59d3f56707af4830a4eda6832b836d89"
format_stamp: "Formatted at 2026-08-12 02:46:13 on dist-test-slave-ss4n"
server_key: "344b386c31b88fef49bd386d85d5aa72"
server_key_iv: "c43f14f0e906fa2a97fb77ae3e66b409"
server_key_version: "encryptionkey@0"
I20260812 02:46:13.282936 1057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:13.303264 1057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:13.308024 1057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:13.308292 1057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:13.328982 1057 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.253:37263
I20260812 02:46:13.329005 1131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.253:37263 every 8 connection(s)
I20260812 02:46:13.330561 1057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
I20260812 02:46:13.332995 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 1057
I20260812 02:46:13.333230 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-1/wal/instance
I20260812 02:46:13.333824 18195 external_mini_cluster.cc:1428] Setting key 1e6112461b92a5c563971247afff8058
I20260812 02:46:13.335188 1132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:13.342211 1132 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.252@KRBTEST.COM" created.
I20260812 02:46:13.369789 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.345130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.17.196.254:44529)
0812 02:46:13.345739 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:13.345746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:13.346933 (+ 1187us) server_negotiation.cc:407] Connection header received
0812 02:46:13.350250 (+ 3317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:13.350255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:13.350342 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:13.350462 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:13.352896 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.353737 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.356421 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.356874 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.356985 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.362408 (+ 5423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:13.362414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:13.362417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:13.362469 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:13.365736 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.366598 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.366603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.366605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.366821 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.367427 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.367432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.367434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.367881 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:13.368077 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:13.368961 (+ 884us) server_negotiation.cc:299] Negotiation successful
0812 02:46:13.369245 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":333,"threads_started":1}
I20260812 02:46:13.370630 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.345768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44529 (local address 127.0.0.1:42170)
0812 02:46:13.346701 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.346760 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:13.349809 (+ 3049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:13.350768 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:13.350784 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:13.351660 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:13.352637 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.352670 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.353928 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.353945 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.356211 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.356240 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.357151 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.357161 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.357550 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.358413 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:13.358453 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:13.362024 (+ 3571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:13.365903 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.365913 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.365931 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.366463 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.366966 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.366972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.366978 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.367290 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.368077 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:13.368085 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:13.368520 (+ 435us) client_negotiation.cc:769] Sending connection context
0812 02:46:13.368819 (+ 299us) client_negotiation.cc:240] Negotiation successful
0812 02:46:13.369285 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":327,"threads_started":1}
I20260812 02:46:13.372709 1132 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:46:13.395074 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.373763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47712)
0812 02:46:13.373979 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.374004 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:13.374133 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:13.375553 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:13.375558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:13.375991 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:13.376205 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.376221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.378619 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.378627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.380466 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.380486 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.382059 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.382100 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.382429 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.383189 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:13.383218 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:13.386437 (+ 3219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:13.391471 (+ 5034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.391480 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.391487 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.392079 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.392618 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.392627 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.392634 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.392837 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.393892 (+ 1055us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:13.393900 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:13.394054 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 02:46:13.394232 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 02:46:13.394570 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 02:46:13.395407 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.374091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.17.196.253:37263)
0812 02:46:13.374911 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:13.374922 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:13.374959 (+ 37us) server_negotiation.cc:407] Connection header received
0812 02:46:13.375019 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:13.375030 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:13.375296 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:13.375461 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:13.376485 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.378442 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.380685 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.381800 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.381955 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.386719 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:13.386740 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:13.386764 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:13.386810 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:13.391277 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.392266 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.392278 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.392283 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.392459 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.392971 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.392978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.392983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.393594 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:13.393824 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:13.394460 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 02:46:13.394783 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":427,"threads_started":1}
I20260812 02:46:13.399094 1132 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:46:13.400647 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.399933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:56946)
0812 02:46:13.400286 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.400452 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260812 02:46:13.400978 1078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
I20260812 02:46:13.403045 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.402649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:56948)
0812 02:46:13.402822 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.402922 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 02:46:13.403467 1132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab.
Entry for principal kudu/127.17.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab.
I20260812 02:46:13.411162 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.252:40057
--webserver_interface=127.17.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab
--principal=kudu/127.17.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44529,127.17.196.253:37263,127.17.196.252:40057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:46:13.452891 1132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 1
I20260812 02:46:13.454217 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.453743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:56964)
0812 02:46:13.453941 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.454053 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 02:46:13.455443 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.455085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:56968)
0812 02:46:13.455259 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.455332 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 02:46:13.455749 1132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
I20260812 02:46:13.505074 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } attempt: 5
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502772, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:46:13.519426 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.505994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.17.196.253:37263)
0812 02:46:13.506188 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:13.506194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:13.506365 (+ 171us) server_negotiation.cc:407] Connection header received
0812 02:46:13.506582 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:13.506593 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:13.506680 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:13.506797 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:13.507666 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.508425 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.509661 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.510178 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.510319 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.513761 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:13.513766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:13.513769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:13.513805 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:13.516532 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.517341 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.517346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.517350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.517529 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.517943 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.517949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.517952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.518268 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:13.518450 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:13.518854 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 02:46:13.519073 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:46:13.519426 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.505920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47722)
0812 02:46:13.506260 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.506279 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:13.506442 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:13.506844 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:13.506852 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:13.507284 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:13.507474 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.507484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.508568 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.508574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.509440 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.509453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.510367 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.510373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.510574 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.511266 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:13.511286 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:13.513536 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:13.516694 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.516699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.516704 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.517195 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.517662 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.517667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.517671 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.517830 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.518394 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:13.518399 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:13.518571 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 02:46:13.518753 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 02:46:13.519053 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260812 02:46:13.520516 1050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:46:13.521708 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.521258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:56984)
0812 02:46:13.521431 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.521569 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 02:46:13.522691 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.522304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57000)
0812 02:46:13.522448 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.522535 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 02:46:13.522940 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
I20260812 02:46:13.523964 1132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 2
I20260812 02:46:13.524837 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.524467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57004)
0812 02:46:13.524627 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.524709 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 02:46:13.525775 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.525429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57010)
0812 02:46:13.525592 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.525673 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260812 02:46:13.526011 1132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
I20260812 02:46:13.574182 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 1
I20260812 02:46:13.575484 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.574995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57018)
0812 02:46:13.575214 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.575312 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 02:46:13.576607 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.576285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57024)
0812 02:46:13.576420 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.576493 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 02:46:13.576911 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
I20260812 02:46:13.617327 1132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 3
I20260812 02:46:13.618765 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.618161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57034)
0812 02:46:13.618402 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.618583 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 02:46:13.619992 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.619509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57040)
0812 02:46:13.619765 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.619862 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260812 02:46:13.620222 1132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
I20260812 02:46:13.645179 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 2
I20260812 02:46:13.646551 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.646031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57046)
0812 02:46:13.646278 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.646376 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260812 02:46:13.647567 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.647211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57062)
0812 02:46:13.647368 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.647454 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 02:46:13.647795 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
I20260812 02:46:13.754070 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 3
I20260812 02:46:13.755313 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.754793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57066)
0812 02:46:13.755028 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.755132 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 02:46:13.756522 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.756106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57082)
0812 02:46:13.756310 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.756402 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260812 02:46:13.756733 1050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
I20260812 02:46:13.763477 1132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 4
I20260812 02:46:13.764778 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.764214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57098)
0812 02:46:13.764474 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.764569 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 02:46:13.766077 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.765563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57100)
0812 02:46:13.765786 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.765910 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 02:46:13.766366 1132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40057: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
W20260812 02:46:13.815398 1144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:13.815760 1144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:13.815928 1144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:13.824441 1144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:13.824600 1144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:13.824697 1144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:13.824796 1144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:13.824883 1144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:13.835758 1144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44529,127.17.196.253:37263,127.17.196.252:40057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab
--principal=kudu/127.17.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.252:40057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:13.837297 1144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:13.838821 1144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:13.852150 1150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:13.852109 1153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:13.852063 1151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:13.853542 1144 server_base.cc:1034] running on GCE node
I20260812 02:46:13.854490 1144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:13.856110 1144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:13.857366 1144 hybrid_clock.cc:648] HybridClock initialized: now 1786502773857318 us; error 41 us; skew 500 ppm
I20260812 02:46:13.857833 1144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:13.861784 1144 init.cc:377] Logged in from keytab as kudu/127.17.196.252@KRBTEST.COM (short username kudu)
I20260812 02:46:13.863750 1144 webserver.cc:460] Webserver started at http://127.17.196.252:38255/ using document root <none> and password file <none>
I20260812 02:46:13.864593 1144 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:13.864770 1144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:13.865125 1144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:13.868817 1144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/data/instance:
uuid: "aeb174247fa244a9b04774a732a65153"
format_stamp: "Formatted at 2026-08-12 02:46:13 on dist-test-slave-ss4n"
server_key: "b3c878040ac6472a0ba5b615dc9a9ca3"
server_key_iv: "9557192ff71c899034ca361565670a82"
server_key_version: "encryptionkey@0"
I20260812 02:46:13.869917 1144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/wal/instance:
uuid: "aeb174247fa244a9b04774a732a65153"
format_stamp: "Formatted at 2026-08-12 02:46:13 on dist-test-slave-ss4n"
server_key: "b3c878040ac6472a0ba5b615dc9a9ca3"
server_key_iv: "9557192ff71c899034ca361565670a82"
server_key_version: "encryptionkey@0"
I20260812 02:46:13.876243 1144 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:46:13.880522 1161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:13.882360 1144 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 02:46:13.882583 1144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/wal
uuid: "aeb174247fa244a9b04774a732a65153"
format_stamp: "Formatted at 2026-08-12 02:46:13 on dist-test-slave-ss4n"
server_key: "b3c878040ac6472a0ba5b615dc9a9ca3"
server_key_iv: "9557192ff71c899034ca361565670a82"
server_key_version: "encryptionkey@0"
I20260812 02:46:13.882817 1144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:13.902002 1144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:13.907919 1144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:13.908458 1144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:13.933964 1050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 4
I20260812 02:46:13.935168 1144 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.252:40057
I20260812 02:46:13.935211 1220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.252:40057 every 8 connection(s)
I20260812 02:46:13.935410 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.934759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57106)
0812 02:46:13.935086 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.935215 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 02:46:13.937214 1144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
I20260812 02:46:13.941790 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 1144
I20260812 02:46:13.942042 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502547576936-18195-0/minicluster-data/master-2/wal/instance
I20260812 02:46:13.942490 1222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:13.942826 18195 external_mini_cluster.cc:1428] Setting key 99e2522e20ec6d00218f9c3ff6b0b689
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502772, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
I20260812 02:46:13.956080 1222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:46:13.965332 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.936269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57116)
0812 02:46:13.936467 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.936491 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:13.936657 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:13.941662 (+ 5005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:13.941669 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:13.942269 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:13.942521 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.942538 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.945355 (+ 2817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.945368 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.947722 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.947750 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.950140 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.950151 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.950495 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.951418 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:13.951470 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:13.955282 (+ 3812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:13.961233 (+ 5951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.961240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.961245 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.961971 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.962699 (+ 728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.962708 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.962712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.962907 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.964126 (+ 1219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:13.964134 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:13.964374 (+ 240us) client_negotiation.cc:769] Sending connection context
0812 02:46:13.964541 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 02:46:13.964881 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 02:46:13.968253 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.937554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.17.196.252:40057)
0812 02:46:13.938733 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:13.938762 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:13.938853 (+ 91us) server_negotiation.cc:407] Connection header received
0812 02:46:13.940713 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:13.940739 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:13.941222 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:13.941705 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:13.942771 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.945082 (+ 2311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.948038 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.949849 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.951127 (+ 1278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.955740 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:13.955754 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:13.955779 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:13.955839 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:13.961012 (+ 5173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.962197 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.962206 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.962217 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.962517 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.963073 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.963081 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.963085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.963933 (+ 848us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:13.965720 (+ 1787us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:13.966057 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 02:46:13.966573 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":812,"thread_start_us":484,"threads_started":1}
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
I20260812 02:46:13.977293 1050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754: Bootstrap starting.
I20260812 02:46:13.980118 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.959771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.17.196.254:44529)
0812 02:46:13.960413 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:13.960420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:13.961527 (+ 1107us) server_negotiation.cc:407] Connection header received
0812 02:46:13.961828 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:13.961833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:13.961927 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:13.962082 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:13.963566 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.964477 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.966530 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.967291 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.967441 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.971961 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:13.971974 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:13.971981 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:13.972032 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:13.975901 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.977028 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.977034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.977037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.977229 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.977881 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.977895 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.977902 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.978407 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:13.978559 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:13.979382 (+ 823us) server_negotiation.cc:299] Negotiation successful
0812 02:46:13.979637 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":337,"threads_started":1}
I20260812 02:46:13.980130 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.960395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44529 (local address 127.0.0.1:42174)
0812 02:46:13.961340 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.961388 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:13.961661 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:13.962127 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:13.962151 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:13.963060 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:13.963406 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.963416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.964696 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.964700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.966308 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.966327 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.967524 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.967531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.967676 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.968469 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:13.968498 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:13.971670 (+ 3172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:13.976087 (+ 4417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.976103 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.976111 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.976853 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.977409 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.977417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.977420 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.977667 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.978674 (+ 1007us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:13.978687 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:13.979023 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 02:46:13.979242 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 02:46:13.979637 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":454,"threads_started":1}
I20260812 02:46:13.981840 1050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:13.982468 1225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.944418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57128 (local address 127.17.196.252:40057)
0812 02:46:13.957352 (+ 12934us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:13.957362 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:13.957396 (+ 34us) server_negotiation.cc:407] Connection header received
0812 02:46:13.957462 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:13.957474 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:13.957588 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:13.959981 (+ 2393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:13.960281 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.961973 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.964273 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.965053 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.965195 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.972824 (+ 7629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:13.972832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:13.972837 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:13.972880 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:13.977030 (+ 4150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.978259 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.978264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.978268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.978439 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.979016 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.979023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.979029 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.979548 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:13.979874 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:13.981726 (+ 1852us) server_negotiation.cc:299] Negotiation successful
0812 02:46:13.981995 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12712,"spinlock_wait_cycles":38656,"thread_start_us":12540,"threads_started":1}
I20260812 02:46:13.982363 1222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:46:13.983389 1050 log.cc:824] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:13.989241 1050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754: No bootstrap required, opened a new log
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
Aug 12 02:46:13 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:13.999559 1050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
I20260812 02:46:13.999997 1050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:14.000165 1050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f237d20ad4744e881703fedd7b26754, State: Initialized, Role: FOLLOWER
I20260812 02:46:14.000684 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.983895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.17.196.253:37263)
0812 02:46:13.984146 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:13.984155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:13.984184 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:46:13.984268 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:13.984277 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:13.984387 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:13.984550 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:13.985425 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.986511 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.988066 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.988598 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.988740 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.992813 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:13.992821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:13.992826 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:13.992888 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:13.996768 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.997552 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.997558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.997560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.997753 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:13.998161 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:13.998167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:13.998171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:13.998448 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:13.998610 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:14.000059 (+ 1449us) server_negotiation.cc:299] Negotiation successful
0812 02:46:14.000343 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 02:46:14.001008 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.983312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:37263 (local address 127.0.0.1:47732)
0812 02:46:13.983976 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:13.984000 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:13.984114 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:13.984588 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:13.984647 (+ 59us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:13.985003 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:13.985165 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.985214 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.986764 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.986770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.987891 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:13.987906 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.988766 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.988772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:13.988983 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.989677 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:13.989695 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:13.992575 (+ 2880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:13.996939 (+ 4364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.996945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.996949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.997388 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.997880 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:13.997885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:13.997889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:13.998041 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:13.998580 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:13.998585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:13.998771 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 02:46:13.998921 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 02:46:13.999263 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515}
I20260812 02:46:14.000980 1050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
I20260812 02:46:14.002357 1222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:46:14.002323 1229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.002771 1229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:46:14.003950 1050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:14.008455 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:13.985748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42190 (local address 127.17.196.254:44529)
0812 02:46:13.985938 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:13.985945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:13.985983 (+ 38us) server_negotiation.cc:407] Connection header received
0812 02:46:13.986038 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:13.986043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:13.986155 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:13.986269 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:13.988758 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.989941 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.991661 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:13.992229 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:13.992384 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:13.998937 (+ 6553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:13.998944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:13.998948 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:13.998991 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:14.004211 (+ 5220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.005243 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.005248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.005252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.005398 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.005972 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.005978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.005982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.006427 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:14.007525 (+ 1098us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:14.007694 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 02:46:14.007945 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 02:46:14 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
W20260812 02:46:14.017763 991 tablet.cc:2369] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:14.018901 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:14.003050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57136)
0812 02:46:14.003220 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:14.003239 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:14.003339 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:14.003819 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:14.003826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:14.004315 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:14.004501 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:14.004510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.005639 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.005644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:14.006945 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:14.006960 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.007884 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.007890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:14.008085 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:14.008791 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:14.008814 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:14.011798 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:14.016083 (+ 4285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:14.016088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:14.016092 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:14.016588 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:14.016990 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:14.016996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:14.016999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:14.017166 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:14.017881 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:14.017886 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:14.018056 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 02:46:14.018205 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 02:46:14.018519 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 02:46:14.019019 1229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:46:14.019436 1229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
I20260812 02:46:14.020006 1225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:14.003272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.17.196.252:40057)
0812 02:46:14.003500 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:14.003508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:14.003529 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:46:14.003583 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:14.003589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:14.003659 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:14.003775 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:14.004643 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.005475 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.007088 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.007724 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.007870 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:14.011986 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:14.011993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:14.012000 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:14.012040 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:14.015946 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.016729 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.016733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.016736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.016884 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.017295 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.017299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.017302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.017748 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:14.019314 (+ 1566us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:14.019434 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 02:46:14.019663 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:46:14.021631 1229 leader_election.cc:288] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59d3f56707af4830a4eda6832b836d89 (127.17.196.253:37263), aeb174247fa244a9b04774a732a65153 (127.17.196.252:40057)
W20260812 02:46:14.024452 1241 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:46:14.024585 1241 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:46:14.026422 1222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153: Bootstrap starting.
I20260812 02:46:14.030769 1222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:14.032258 1222 log.cc:824] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:46:14 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:46:14.023686 966 master_runner.cc:186] Error getting master registration for 127.17.196.253:37263: OK, instance_id { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" instance_seqno: 1786502773323025 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:46:14.037740 966 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:46:14.041849 1222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153: No bootstrap required, opened a new log
I20260812 02:46:14.046460 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:14.010954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47734 (local address 127.17.196.253:37263)
0812 02:46:14.011141 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:14.011147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:14.011171 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:46:14.011272 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:14.011280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:14.011359 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:14.011481 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:14.013372 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.014715 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.016721 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.017462 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.018734 (+ 1272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:14.038010 (+ 19276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:14.038018 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:14.038023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:14.038083 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:14.042285 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.043522 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.043529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.043533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.043852 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.044391 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.044396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.044400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.044862 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:14.045156 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:14.045631 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 02:46:14.045882 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 02:46:14.052345 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:46:14.055225 1222 raft_consensus.cc:348] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
I20260812 02:46:14.055774 1222 raft_consensus.cc:374] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:14.055930 1222 raft_consensus.cc:729] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb174247fa244a9b04774a732a65153, State: Initialized, Role: FOLLOWER
I20260812 02:46:14.056708 1222 consensus_queue.cc:260] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
I20260812 02:46:14.065557 1132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } attempt: 5
I20260812 02:46:14.067826 1245 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.068310 1245 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:46:14.068789 1222 sys_catalog.cc:564] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:14.068822 1193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0f237d20ad4744e881703fedd7b26754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeb174247fa244a9b04774a732a65153" is_pre_election: true
I20260812 02:46:14.069881 1193 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f237d20ad4744e881703fedd7b26754 in term 0.
I20260812 02:46:14.070838 988 leader_election.cc:302] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f237d20ad4744e881703fedd7b26754, aeb174247fa244a9b04774a732a65153; no voters:
I20260812 02:46:14.071354 1229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:46:14.071580 1229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:46:14.071715 1229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:14.076524 1229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
Aug 12 02:46:14 dist-test-slave-ss4n krb5kdc[950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
I20260812 02:46:14.079270 1193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0f237d20ad4744e881703fedd7b26754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeb174247fa244a9b04774a732a65153"
I20260812 02:46:14.079721 1193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:14.084950 1193 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f237d20ad4744e881703fedd7b26754 in term 1.
I20260812 02:46:14.085299 1229 leader_election.cc:288] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [CANDIDATE]: Term 1 election: Requested vote from peers 59d3f56707af4830a4eda6832b836d89 (127.17.196.253:37263), aeb174247fa244a9b04774a732a65153 (127.17.196.252:40057)
I20260812 02:46:14.086457 988 leader_election.cc:302] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f237d20ad4744e881703fedd7b26754, aeb174247fa244a9b04774a732a65153; no voters:
I20260812 02:46:14.087147 1229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:14.089669 1229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [term 1 LEADER]: Becoming Leader. State: Replica: 0f237d20ad4744e881703fedd7b26754, State: Running, Role: LEADER
I20260812 02:46:14.091143 1229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
I20260812 02:46:14.094413 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:14.066419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40057 (local address 127.0.0.1:57142)
0812 02:46:14.066890 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:14.066913 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:14.067050 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:14.067450 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:14.067456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:14.068053 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:14.068298 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:14.068314 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.070376 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.070384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:14.072029 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:14.072053 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.073583 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.073593 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:14.073884 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:14.074648 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:14.074673 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:14.077649 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:14.089886 (+ 12237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:14.089897 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:14.089903 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:14.090519 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:14.091436 (+ 917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:14.091442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:14.091445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:14.091675 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:14.092908 (+ 1233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:14.092917 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:14.093167 (+ 250us) client_negotiation.cc:769] Sending connection context
0812 02:46:14.093340 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 02:46:14.093816 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322}
I20260812 02:46:14.097656 1225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:14.066841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57142 (local address 127.17.196.252:40057)
0812 02:46:14.067024 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:14.067032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:14.067060 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:46:14.067186 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:14.067191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:14.067282 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:14.067403 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:14.068462 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.070120 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.072272 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.073068 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.073241 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:14.082660 (+ 9419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:14.082668 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:14.082675 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:14.082786 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:14.089582 (+ 6796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.090815 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.090820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.090824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.091208 (+ 384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.091927 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.091934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.091939 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.092688 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:14.095493 (+ 2805us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:14.095893 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 02:46:14.096522 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 02:46:14.106709 1132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89: Bootstrap starting.
I20260812 02:46:14.124173 1132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:14.130142 1252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f237d20ad4744e881703fedd7b26754. Latest consensus state: current_term: 1 leader_uuid: "0f237d20ad4744e881703fedd7b26754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.130713 1252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:14.133888 1132 log.cc:824] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:14.143148 1265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:46:14.150722 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:14.100892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.17.196.254:44529)
0812 02:46:14.101093 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:14.101099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:14.107052 (+ 5953us) server_negotiation.cc:407] Connection header received
0812 02:46:14.107836 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:14.107842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:14.107925 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:14.108001 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:14.115433 (+ 7432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.116618 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.121906 (+ 5288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.122656 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.122770 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:14.135768 (+ 12998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:14.135778 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:14.135784 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:14.135832 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:14.140071 (+ 4239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.143026 (+ 2955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.143032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.143036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.143219 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.143780 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.143789 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.143792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.144273 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:14.144570 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:14.150043 (+ 5473us) server_negotiation.cc:299] Negotiation successful
0812 02:46:14.150314 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:46:14.152575 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:14.092594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47736 (local address 127.17.196.253:37263)
0812 02:46:14.092801 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:14.092809 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:14.104881 (+ 12072us) server_negotiation.cc:407] Connection header received
0812 02:46:14.105448 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:14.105454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:14.105588 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:14.105677 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:14.110995 (+ 5318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.112251 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.119591 (+ 7340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.120306 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.120421 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:14.141210 (+ 20789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:14.141218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:14.141223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:14.141274 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:14.146020 (+ 4746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.147082 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.147090 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.147095 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.147286 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.147823 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.147830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.147836 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.148247 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:14.148507 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:14.151940 (+ 3433us) server_negotiation.cc:299] Negotiation successful
0812 02:46:14.152211 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:46:14.152916 1132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89: No bootstrap required, opened a new log
I20260812 02:46:14.154788 1265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:46:14.155176 1267 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260812 02:46:14.161631 1168 tablet.cc:2369] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:14.169348 1132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
I20260812 02:46:14.169901 1132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:14.170079 1132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d3f56707af4830a4eda6832b836d89, State: Initialized, Role: FOLLOWER
I20260812 02:46:14.170864 1132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } }
I20260812 02:46:14.162853 1144 master_runner.cc:186] Error getting master registration for 127.17.196.253:37263: OK, instance_id { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" instance_seqno: 1786502773323025 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:46:14.176324 1144 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:46:14.177274 1225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:14.114913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57158 (local address 127.17.196.252:40057)
0812 02:46:14.115138 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:14.115148 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:14.124420 (+ 9272us) server_negotiation.cc:407] Connection header received
0812 02:46:14.130144 (+ 5724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:14.130150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:14.130251 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:14.130343 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:14.132465 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.133710 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.145161 (+ 11451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:14.146008 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:14.146133 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:14.158092 (+ 11959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:14.158100 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:14.158104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:14.158150 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:14.161994 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.174146 (+ 12152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.174152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.174156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.174339 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:14.174953 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:14.174962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:14.174968 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:14.175411 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:14.175676 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:14.176649 (+ 973us) server_negotiation.cc:299] Negotiation successful
0812 02:46:14.176917 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 02:46:14.178045 1268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.178503 1268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:46:14.180600 1132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:14.180635 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0f237d20ad4744e881703fedd7b26754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59d3f56707af4830a4eda6832b836d89"
I20260812 02:46:14.181156 1103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:14.185644 1104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0f237d20ad4744e881703fedd7b26754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59d3f56707af4830a4eda6832b836d89" is_pre_election: true
I20260812 02:46:14.186427 1103 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f237d20ad4744e881703fedd7b26754 in term 1.
I20260812 02:46:14.190783 1193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [term 1 FOLLOWER]: Refusing update from remote peer 0f237d20ad4744e881703fedd7b26754: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:14.191859 1252 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:14.192471 1103 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [term 1 FOLLOWER]: Refusing update from remote peer 0f237d20ad4744e881703fedd7b26754: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:14.193223 1252 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:14.210376 1268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f237d20ad4744e881703fedd7b26754. Latest consensus state: current_term: 1 leader_uuid: "0f237d20ad4744e881703fedd7b26754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.210783 1268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:46:14.216040 1269 mvcc.cc:205] Tried to move back new op lower bound from 7317515363070836736 to 7317515362691952640. Current Snapshot: MvccSnapshot[applied={T|T < 7317515363070836736}]
I20260812 02:46:14.218724 1252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f237d20ad4744e881703fedd7b26754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.219074 1252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:14.219916 1268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f237d20ad4744e881703fedd7b26754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.220204 1268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:46:14.221060 1274 mvcc.cc:205] Tried to move back new op lower bound from 7317515363070836736 to 7317515362691952640. Current Snapshot: MvccSnapshot[applied={T|T < 7317515363070836736 or (T in {7317515363070836736})}]
I20260812 02:46:14.224778 1245 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f237d20ad4744e881703fedd7b26754. Latest consensus state: current_term: 1 leader_uuid: "0f237d20ad4744e881703fedd7b26754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.224826 1229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f237d20ad4744e881703fedd7b26754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.225123 1245 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:46:14.225173 1229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:14.226728 1265 catalog_manager.cc:1347] Generated new cluster ID: 0f7232c963a04703ad63248dc037f88a
I20260812 02:46:14.226856 1265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:46:14.229746 1273 mvcc.cc:205] Tried to move back new op lower bound from 7317515363070836736 to 7317515362691952640. Current Snapshot: MvccSnapshot[applied={T|T < 7317515363070836736}]
I20260812 02:46:14.231274 1245 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f237d20ad4744e881703fedd7b26754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f237d20ad4744e881703fedd7b26754" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44529 } } peers { permanent_uuid: "59d3f56707af4830a4eda6832b836d89" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 37263 } } peers { permanent_uuid: "aeb174247fa244a9b04774a732a65153" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40057 } } }
I20260812 02:46:14.231580 1245 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:46:14.256592 1265 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:46:14.259387 1265 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:46:14.280879 1265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f237d20ad4744e881703fedd7b26754: Generated new TSK 0
I20260812 02:46:14.282217 1265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:46:14.283609 1294 catalog_manager.cc:1259] Loaded cluster ID: 0f7232c963a04703ad63248dc037f88a
I20260812 02:46:14.286918 1294 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89: loading cluster ID for follower catalog manager: success
I20260812 02:46:14.291155 1294 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89: acquiring CA information for follower catalog manager: success
I20260812 02:46:14.294096 1294 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59d3f56707af4830a4eda6832b836d89: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:46:15.160486 1267 catalog_manager.cc:1259] Loaded cluster ID: 0f7232c963a04703ad63248dc037f88a
I20260812 02:46:15.160646 1267 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153: loading cluster ID for follower catalog manager: success
I20260812 02:46:15.163506 1267 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153: acquiring CA information for follower catalog manager: success
I20260812 02:46:15.165889 1267 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aeb174247fa244a9b04774a732a65153: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:46:15.382400 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 966
I20260812 02:46:15.383899 1044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:15.501235 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 966
W20260812 02:46:15.528355 1076 connection.cc:535] server connection from 127.0.0.1:47722 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:46:15.528650 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 1057
I20260812 02:46:15.530010 1127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:15.640411 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 1057
I20260812 02:46:15.666038 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 1144
I20260812 02:46:15.667335 1215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:15.777732 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 1144
2026-08-12T02:46:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5044 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:46:15 dist-test-slave-ss4n krb5kdc[1308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:46:15 dist-test-slave-ss4n krb5kdc[1308](info): set up 2 sockets
Aug 12 02:46:15 dist-test-slave-ss4n krb5kdc[1308](info): commencing operation
krb5kdc: starting...
W20260812 02:46:17.061993 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:46:17 dist-test-slave-ss4n krb5kdc[1308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:46:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:46:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:46:17.377995 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:34701
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:34701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:17.740666 1323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:17.740985 1323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:17.741077 1323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:17.748808 1323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:17.748937 1323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:17.748987 1323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:17.749040 1323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:17.749087 1323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:17.759529 1323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:34701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:34701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:17.760921 1323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:17.762413 1323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:17.775264 1328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:17.775314 1331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:17.775499 1329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:17.776598 1323 server_base.cc:1034] running on GCE node
I20260812 02:46:17.777555 1323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:17.779232 1323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:17.780525 1323 hybrid_clock.cc:648] HybridClock initialized: now 1786502777780427 us; error 43 us; skew 500 ppm
I20260812 02:46:17.780870 1323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:17 dist-test-slave-ss4n krb5kdc[1308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502777, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:17.787847 1323 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:46:17.790015 1323 webserver.cc:460] Webserver started at http://127.17.196.254:39437/ using document root <none> and password file <none>
I20260812 02:46:17.790827 1323 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:17.790972 1323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:17.791280 1323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:17.794930 1323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f"
format_stamp: "Formatted at 2026-08-12 02:46:17 on dist-test-slave-ss4n"
server_key: "a600f84417868d6328b598ee77fbc026"
server_key_iv: "2a8a0bcad903659d8f949eec7fc4d2b4"
server_key_version: "encryptionkey@0"
I20260812 02:46:17.795785 1323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f"
format_stamp: "Formatted at 2026-08-12 02:46:17 on dist-test-slave-ss4n"
server_key: "a600f84417868d6328b598ee77fbc026"
server_key_iv: "2a8a0bcad903659d8f949eec7fc4d2b4"
server_key_version: "encryptionkey@0"
I20260812 02:46:17.802238 1323 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260812 02:46:17.806449 1338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:17.808338 1323 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260812 02:46:17.808590 1323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f"
format_stamp: "Formatted at 2026-08-12 02:46:17 on dist-test-slave-ss4n"
server_key: "a600f84417868d6328b598ee77fbc026"
server_key_iv: "2a8a0bcad903659d8f949eec7fc4d2b4"
server_key_version: "encryptionkey@0"
I20260812 02:46:17.808847 1323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:17.843240 1323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:17.848505 1323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:17.848773 1323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:17.870213 1323 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:34701
I20260812 02:46:17.870220 1397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:34701 every 8 connection(s)
I20260812 02:46:17.871837 1323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:46:17.875937 1398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:17.880327 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 1323
I20260812 02:46:17.880576 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:46:17.881423 18195 external_mini_cluster.cc:1428] Setting key 8c2ad26e3daca749029fb2c45dd1ea0c
I20260812 02:46:17.888525 1398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f: Bootstrap starting.
I20260812 02:46:17.892405 1398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:17.893981 1398 log.cc:824] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:17.898239 1398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f: No bootstrap required, opened a new log
Aug 12 02:46:17 dist-test-slave-ss4n krb5kdc[1308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:17.906786 1398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34701 } }
I20260812 02:46:17.907256 1398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:17.907406 1398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dee59ce742a4aa49f42c4c6ceb84a0f, State: Initialized, Role: FOLLOWER
I20260812 02:46:17.908211 1398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34701 } }
I20260812 02:46:17.908638 1398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:46:17.908802 1398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:46:17.909005 1398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:17.909333 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:17.883778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.17.196.254:34701)
0812 02:46:17.884838 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:17.884851 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:17.884898 (+ 47us) server_negotiation.cc:407] Connection header received
0812 02:46:17.887112 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:17.887141 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:17.887584 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:17.887949 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:17.889997 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:17.891624 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:17.893796 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:17.894904 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:17.895182 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:17.900246 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:17.900255 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:17.900274 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:17.900319 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:17.903963 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:17.904883 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:17.904895 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:17.904903 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:17.905112 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:17.905659 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:17.905663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:17.905665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:17.906320 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:17.906679 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:17.907253 (+ 574us) server_negotiation.cc:299] Negotiation successful
0812 02:46:17.907781 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":425,"threads_started":1}
I20260812 02:46:17.912200 1398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34701 } }
I20260812 02:46:17.912832 1398 leader_election.cc:302] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dee59ce742a4aa49f42c4c6ceb84a0f; no voters:
I20260812 02:46:17.913825 1398 leader_election.cc:288] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:46:17.914202 1404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:17.914978 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:46:17.922530 1404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [term 1 LEADER]: Becoming Leader. State: Replica: 8dee59ce742a4aa49f42c4c6ceb84a0f, State: Running, Role: LEADER
I20260812 02:46:17.922859 1398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:17.923218 1404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34701 } }
I20260812 02:46:17.927435 1405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34701 } } }
I20260812 02:46:17.927827 1405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:17.928771 1412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:46:17.928686 1406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dee59ce742a4aa49f42c4c6ceb84a0f. Latest consensus state: current_term: 1 leader_uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee59ce742a4aa49f42c4c6ceb84a0f" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34701 } } }
I20260812 02:46:17.929013 1406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:17.936837 1412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:46:17.946118 1412 catalog_manager.cc:1347] Generated new cluster ID: 983dd28217a54fff9fdf0397cbfef930
I20260812 02:46:17.946259 1412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:46:17.968356 1412 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:46:17.969874 1412 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:46:17.978305 1412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dee59ce742a4aa49f42c4c6ceb84a0f: Generated new TSK 0
I20260812 02:46:17.979171 1412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:46:18.031203 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 1323
I20260812 02:46:18.032583 1393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:18.134434 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 1323
2026-08-12T02:46:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2353 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:46:18 dist-test-slave-ss4n krb5kdc[1427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:46:18 dist-test-slave-ss4n krb5kdc[1427](info): set up 2 sockets
Aug 12 02:46:18 dist-test-slave-ss4n krb5kdc[1427](info): commencing operation
krb5kdc: starting...
W20260812 02:46:19.434448 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:46:19 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:46:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:46:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:46:19.754845 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37967
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:37967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:20.123147 1440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:20.123561 1440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:20.123746 1440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:20.131883 1440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:20.132077 1440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:20.132189 1440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:20.132295 1440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:20.132382 1440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:20.142817 1440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:37967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:20.144309 1440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:20.145752 1440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:20.159624 1447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:20.159710 1446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:20.159664 1449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:20.160687 1440 server_base.cc:1034] running on GCE node
I20260812 02:46:20.161701 1440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:20.163512 1440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:20.164748 1440 hybrid_clock.cc:648] HybridClock initialized: now 1786502780164683 us; error 63 us; skew 500 ppm
I20260812 02:46:20.165230 1440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:20 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:20.168973 1440 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:46:20.170697 1440 webserver.cc:460] Webserver started at http://127.17.196.254:36989/ using document root <none> and password file <none>
I20260812 02:46:20.171483 1440 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:20.171617 1440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:20.171917 1440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:20.175253 1440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "ad8407290538402c61b32f80c0bfc594"
server_key_iv: "320f8d174ca2a83ae61f208b43572059"
server_key_version: "encryptionkey@0"
I20260812 02:46:20.176175 1440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "ad8407290538402c61b32f80c0bfc594"
server_key_iv: "320f8d174ca2a83ae61f208b43572059"
server_key_version: "encryptionkey@0"
I20260812 02:46:20.182228 1440 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:46:20.186323 1458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:20.187904 1440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:46:20.188130 1440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "ad8407290538402c61b32f80c0bfc594"
server_key_iv: "320f8d174ca2a83ae61f208b43572059"
server_key_version: "encryptionkey@0"
I20260812 02:46:20.188366 1440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:20.212002 1440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:20.217052 1440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:20.217392 1440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:20.240350 1440 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:37967
I20260812 02:46:20.240396 1516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:37967 every 8 connection(s)
I20260812 02:46:20.242033 1440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:46:20.246361 1517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:20.247352 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 1440
I20260812 02:46:20.247627 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:46:20.248428 18195 external_mini_cluster.cc:1428] Setting key 87ae2d032f126a064b9905aaea95efbe
I20260812 02:46:20.261246 1517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Bootstrap starting.
Aug 12 02:46:20 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:20.265429 1517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:20.266791 1517 log.cc:824] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:20.270393 1517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: No bootstrap required, opened a new log
I20260812 02:46:20.274672 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:20.250747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.17.196.254:37967)
0812 02:46:20.251705 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:20.251724 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:20.252105 (+ 381us) server_negotiation.cc:407] Connection header received
0812 02:46:20.254145 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:20.254170 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:20.254649 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:20.255110 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:20.256802 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:20.258425 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:20.259971 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:20.260738 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:20.260985 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:20.265985 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:20.265996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:20.266022 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:20.266069 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:20.269819 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:20.270726 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:20.270734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:20.270742 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:20.270964 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:20.271433 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:20.271439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:20.271443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:20.272062 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:20.272469 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:20.272985 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 02:46:20.273465 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":408,"threads_started":1}
I20260812 02:46:20.279206 1517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:20.279628 1517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:20.279738 1517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bb2a80fef614c0cbfdf7f0670da6ad7, State: Initialized, Role: FOLLOWER
I20260812 02:46:20.280328 1517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:20.280660 1517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:46:20.280840 1517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:46:20.281018 1517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:20.283527 1517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:20.284196 1517 leader_election.cc:302] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bb2a80fef614c0cbfdf7f0670da6ad7; no voters:
I20260812 02:46:20.284977 1517 leader_election.cc:288] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:46:20.285132 1522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:20.285916 1522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 1 LEADER]: Becoming Leader. State: Replica: 2bb2a80fef614c0cbfdf7f0670da6ad7, State: Running, Role: LEADER
I20260812 02:46:20.286614 1522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:20.287526 1517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:20.290159 1523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } } }
I20260812 02:46:20.290122 1524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb2a80fef614c0cbfdf7f0670da6ad7. Latest consensus state: current_term: 1 leader_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } } }
I20260812 02:46:20.290660 1523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:20.290673 1524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:20.298779 1529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:46:20.303635 1529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:46:20.313552 1529 catalog_manager.cc:1347] Generated new cluster ID: a5969e16c29b48c797e14c2acf035473
I20260812 02:46:20.313735 1529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:46:20.323045 1529 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:46:20.324543 1529 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:46:20.336737 1529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Generated new TSK 0
I20260812 02:46:20.338091 1529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:46:20.418638 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37967
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:20.803656 1549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:20.804010 1549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:20.804185 1549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:20.811654 1549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:20.811818 1549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:20.812008 1549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:46:20.821938 1549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:20.823374 1549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:20.824765 1549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:20.838738 1558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:20.838773 1556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:20.838807 1554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:20.839697 1549 server_base.cc:1034] running on GCE node
I20260812 02:46:20.840426 1549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:20.841405 1549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:20.842677 1549 hybrid_clock.cc:648] HybridClock initialized: now 1786502780842637 us; error 76 us; skew 500 ppm
I20260812 02:46:20.843024 1549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:20 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:20.846642 1549 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:46:20.848203 1549 webserver.cc:460] Webserver started at http://127.17.196.193:38289/ using document root <none> and password file <none>
I20260812 02:46:20.848958 1549 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:20.849141 1549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:20.849525 1549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:20.852705 1549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "655222ec0c32432993a1b9dea125c5d1"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "39620fcec97e00bc8cad70b80f99177e"
server_key_iv: "9168d26243cc1647c7cdb8ae23812f48"
server_key_version: "encryptionkey@0"
I20260812 02:46:20.853605 1549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "655222ec0c32432993a1b9dea125c5d1"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "39620fcec97e00bc8cad70b80f99177e"
server_key_iv: "9168d26243cc1647c7cdb8ae23812f48"
server_key_version: "encryptionkey@0"
I20260812 02:46:20.859277 1549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 02:46:20.863257 1568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:20.864879 1549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:46:20.865118 1549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "655222ec0c32432993a1b9dea125c5d1"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "39620fcec97e00bc8cad70b80f99177e"
server_key_iv: "9168d26243cc1647c7cdb8ae23812f48"
server_key_version: "encryptionkey@0"
I20260812 02:46:20.865378 1549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:20.895025 1549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:20.900354 1549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:20.900719 1549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:20.902021 1549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:20.903780 1549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:20.903910 1549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:20.904090 1549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:20.904172 1549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:20.945102 1549 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:43881
I20260812 02:46:20.945205 1694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:43881 every 8 connection(s)
I20260812 02:46:20.946645 1549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:46:20.953979 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 1549
I20260812 02:46:20.954214 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:46:20.955050 18195 external_mini_cluster.cc:1428] Setting key 134825e4e3542a96a6875a9225b33d54
Aug 12 02:46:20 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:20.969592 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:20.949561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:44579 (local address 127.17.196.254:37967)
0812 02:46:20.950097 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:20.950103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:20.951168 (+ 1065us) server_negotiation.cc:407] Connection header received
0812 02:46:20.953241 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:20.953248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:20.953358 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:20.953541 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:20.955512 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:20.956381 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:20.958256 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:20.958714 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:20.958880 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:20.963083 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:20.963089 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:20.963092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:20.963127 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:20.966110 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:20.966959 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:20.966964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:20.966967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:20.967098 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:20.967597 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:20.967602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:20.967604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:20.967903 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:20.968077 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:20.968862 (+ 785us) server_negotiation.cc:299] Negotiation successful
0812 02:46:20.969124 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":263,"threads_started":1}
I20260812 02:46:20.970862 1699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:20.950146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:44579)
0812 02:46:20.950979 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:20.951028 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:20.952937 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:20.953811 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:20.953820 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:20.954563 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:20.955306 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:20.955328 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:20.956583 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:20.956593 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:20.958079 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:20.958093 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:20.958993 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:20.959004 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:20.959270 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:20.960063 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:20.960089 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:20.962783 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:20.966272 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:20.966282 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:20.966301 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:20.966791 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:20.967226 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:20.967231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:20.967236 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:20.967457 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:20.968053 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:20.968062 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:20.968441 (+ 379us) client_negotiation.cc:769] Sending connection context
0812 02:46:20.968737 (+ 296us) client_negotiation.cc:240] Negotiation successful
0812 02:46:20.969389 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":335,"threads_started":1}
I20260812 02:46:20.972782 1695 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37967
I20260812 02:46:20.973161 1695 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:20.974067 1695 heartbeater.cc:511] Master 127.17.196.254:37967 requested a full tablet report, sending...
I20260812 02:46:20.976701 1478 ts_manager.cc:194] Registered new tserver with Master: 655222ec0c32432993a1b9dea125c5d1 (127.17.196.193:43881)
I20260812 02:46:20.979285 1478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:44579
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:46:21.027608 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37967
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:21.385794 1702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:21.386196 1702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:21.386385 1702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:21.395597 1702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:21.395742 1702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:21.395977 1702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:46:21.406859 1702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:21.408317 1702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:21.409775 1702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:21.424060 1712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:21.424139 1709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:21.424199 1708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:21.425186 1702 server_base.cc:1034] running on GCE node
I20260812 02:46:21.426016 1702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:21.427104 1702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:21.428337 1702 hybrid_clock.cc:648] HybridClock initialized: now 1786502781428279 us; error 51 us; skew 500 ppm
I20260812 02:46:21.428684 1702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:21 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:21.432698 1702 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:46:21.434509 1702 webserver.cc:460] Webserver started at http://127.17.196.194:46391/ using document root <none> and password file <none>
I20260812 02:46:21.435369 1702 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:21.435531 1702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:21.435900 1702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:21.439424 1702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "a092adcdb0434969b20f9350482e75b7"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "c03f32e25e1c1cbefd0163dac1ac34de"
server_key_iv: "fde051860bb81ae434a5dcddae8812af"
server_key_version: "encryptionkey@0"
I20260812 02:46:21.440352 1702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "a092adcdb0434969b20f9350482e75b7"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "c03f32e25e1c1cbefd0163dac1ac34de"
server_key_iv: "fde051860bb81ae434a5dcddae8812af"
server_key_version: "encryptionkey@0"
I20260812 02:46:21.446501 1702 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 02:46:21.450730 1719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:21.452293 1702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:46:21.452520 1702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "a092adcdb0434969b20f9350482e75b7"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "c03f32e25e1c1cbefd0163dac1ac34de"
server_key_iv: "fde051860bb81ae434a5dcddae8812af"
server_key_version: "encryptionkey@0"
I20260812 02:46:21.452773 1702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:21.465319 1702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:21.470345 1702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:21.470724 1702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:21.471879 1702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:21.473336 1702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:21.473441 1702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:21.473613 1702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:21.473690 1702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:21.510442 1702 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:46121
I20260812 02:46:21.510514 1846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:46121 every 8 connection(s)
I20260812 02:46:21.511811 1702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:46:21.519999 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 1702
I20260812 02:46:21.520198 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:46:21.520707 18195 external_mini_cluster.cc:1428] Setting key ea1518c874363694d72b49f0eb861ef4
Aug 12 02:46:21 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:21.533304 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:21.514884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:56727 (local address 127.17.196.254:37967)
0812 02:46:21.515384 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:21.515389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:21.516205 (+ 816us) server_negotiation.cc:407] Connection header received
0812 02:46:21.518238 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:21.518245 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:21.518344 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:21.518469 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:21.520043 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:21.520969 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:21.522634 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:21.523097 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:21.523231 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:21.527124 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:21.527129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:21.527132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:21.527166 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:21.530046 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:21.530849 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:21.530854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:21.530856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:21.530986 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:21.531452 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:21.531464 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:21.531467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:21.531742 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:21.531922 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:21.532661 (+ 739us) server_negotiation.cc:299] Negotiation successful
0812 02:46:21.532905 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":276,"threads_started":1}
I20260812 02:46:21.534376 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:21.515258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:56727)
0812 02:46:21.516032 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:21.516071 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:21.517948 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:21.518732 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:21.518741 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:21.519272 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:21.519848 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:21.519866 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:21.521141 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:21.521145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:21.522466 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:21.522483 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:21.523343 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:21.523350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:21.523594 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:21.524326 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:21.524350 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:21.526847 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:21.530212 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:21.530222 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:21.530241 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:21.530690 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:21.531106 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:21.531111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:21.531115 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:21.531326 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:21.531878 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:21.531888 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:21.532274 (+ 386us) client_negotiation.cc:769] Sending connection context
0812 02:46:21.532563 (+ 289us) client_negotiation.cc:240] Negotiation successful
0812 02:46:21.533153 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":325,"threads_started":1}
I20260812 02:46:21.536058 1847 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37967
I20260812 02:46:21.536401 1847 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:21.537168 1847 heartbeater.cc:511] Master 127.17.196.254:37967 requested a full tablet report, sending...
I20260812 02:46:21.538815 1478 ts_manager.cc:194] Registered new tserver with Master: a092adcdb0434969b20f9350482e75b7 (127.17.196.194:46121)
I20260812 02:46:21.539799 1478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:56727
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:46:21.590042 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37967
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:21.949091 1855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:21.949445 1855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:21.949646 1855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:21.957909 1855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:21.958088 1855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:21.958287 1855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:46:21.968626 1855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:21.970278 1855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:21.971743 1855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:46:21.982846 1695 heartbeater.cc:503] Master 127.17.196.254:37967 was elected leader, sending a full tablet report...
W20260812 02:46:21.987154 1860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:21.987190 1864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:21.987100 1862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:21.988135 1855 server_base.cc:1034] running on GCE node
I20260812 02:46:21.988873 1855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:21.990003 1855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:21.991251 1855 hybrid_clock.cc:648] HybridClock initialized: now 1786502781991194 us; error 54 us; skew 500 ppm
I20260812 02:46:21.991621 1855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:21 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:21.995501 1855 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:46:21.997174 1855 webserver.cc:460] Webserver started at http://127.17.196.195:38391/ using document root <none> and password file <none>
I20260812 02:46:21.997956 1855 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:21.998102 1855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:21.998430 1855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:22.001636 1855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "6df02ad79c3847af92272dfc3c64bad2"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "da3320dd20b745ae73a95dafdf5b71e6"
server_key_iv: "211cd0c66d163a8742a2594bdda28ee5"
server_key_version: "encryptionkey@0"
I20260812 02:46:22.002485 1855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "6df02ad79c3847af92272dfc3c64bad2"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "da3320dd20b745ae73a95dafdf5b71e6"
server_key_iv: "211cd0c66d163a8742a2594bdda28ee5"
server_key_version: "encryptionkey@0"
I20260812 02:46:22.007999 1855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:46:22.012223 1875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:22.013942 1855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:46:22.014165 1855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "6df02ad79c3847af92272dfc3c64bad2"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "da3320dd20b745ae73a95dafdf5b71e6"
server_key_iv: "211cd0c66d163a8742a2594bdda28ee5"
server_key_version: "encryptionkey@0"
I20260812 02:46:22.014372 1855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:22.047962 1855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:22.052994 1855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:22.053371 1855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:22.054459 1855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:22.055943 1855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:22.056020 1855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:22.056249 1855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:22.056330 1855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:22.094902 1855 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:42517
I20260812 02:46:22.094960 2008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:42517 every 8 connection(s)
I20260812 02:46:22.096278 1855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:46:22.102764 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 1855
I20260812 02:46:22.103006 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:46:22.103504 18195 external_mini_cluster.cc:1428] Setting key f0190af70a9d6f8459837785f5715bcc
Aug 12 02:46:22 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:22.120555 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.099186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:42285 (local address 127.17.196.254:37967)
0812 02:46:22.099695 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.099700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.100654 (+ 954us) server_negotiation.cc:407] Connection header received
0812 02:46:22.102687 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.102693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.102808 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.102935 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:22.104774 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.105853 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.107657 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.108085 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.108224 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.112529 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:22.112537 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:22.112542 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:22.112592 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:22.116443 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.117473 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.117534 (+ 61us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.117541 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.117736 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.118352 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.118356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.118359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.118866 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:22.119123 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.120018 (+ 895us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.120224 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":267,"threads_started":1}
I20260812 02:46:22.121693 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.099647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:42285)
0812 02:46:22.100405 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:22.100449 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:22.102373 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:22.103208 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:22.103217 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:22.103832 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:22.104580 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.104604 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.106026 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.106030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.107484 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.107497 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.108331 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.108341 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.108603 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.109348 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:22.109379 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:22.112275 (+ 2896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:22.116699 (+ 4424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:22.116708 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:22.116730 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:22.117304 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:22.117910 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:22.117915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:22.117938 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:22.118207 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:22.119053 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:22.119063 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:22.119540 (+ 477us) client_negotiation.cc:769] Sending connection context
0812 02:46:22.119926 (+ 386us) client_negotiation.cc:240] Negotiation successful
0812 02:46:22.120420 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":332,"threads_started":1}
I20260812 02:46:22.123106 2010 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37967
I20260812 02:46:22.123386 2010 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:22.124152 2010 heartbeater.cc:511] Master 127.17.196.254:37967 requested a full tablet report, sending...
I20260812 02:46:22.125694 1478 ts_manager.cc:194] Registered new tserver with Master: 6df02ad79c3847af92272dfc3c64bad2 (127.17.196.195:42517)
I20260812 02:46:22.126570 1478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:42285
I20260812 02:46:22.132388 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:46:22.151974 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.137229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.17.196.254:37967)
0812 02:46:22.137547 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.137555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.137910 (+ 355us) server_negotiation.cc:407] Connection header received
0812 02:46:22.138157 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.138162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.138247 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.138348 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:22.140109 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.140988 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.142468 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.143012 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.143124 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.145810 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:22.145815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:22.145819 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:22.145858 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:22.148904 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.149706 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.149711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.149713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.149849 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.150348 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.150358 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.150361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.150867 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:22.151086 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.151478 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.151720 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260812 02:46:22.157536 1478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:46:22.165254 1478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:46:22.193042 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.179942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:46121 (local address 127.0.0.1:34030)
0812 02:46:22.182086 (+ 2144us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:22.182108 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:22.182249 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:22.183877 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:22.183889 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:22.183933 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:22.184541 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.184559 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.187144 (+ 2585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.187151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.189950 (+ 2799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.189962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.191884 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.191891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.192133 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.192191 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 02:46:22.192327 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 02:46:22.192624 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1984,"thread_start_us":5479,"threads_started":1}
I20260812 02:46:22.193610 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.180305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34030 (local address 127.17.196.194:46121)
0812 02:46:22.182761 (+ 2456us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.182773 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.182812 (+ 39us) server_negotiation.cc:407] Connection header received
0812 02:46:22.182937 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.182948 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.183289 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.183525 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:22.185303 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.186977 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.190124 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.191661 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.192692 (+ 1031us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.192930 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.193009 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.193212 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2286,"thread_start_us":2167,"threads_started":1}
I20260812 02:46:22.194006 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.178730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:42517 (local address 127.0.0.1:50956)
0812 02:46:22.180397 (+ 1667us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:22.180438 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:22.180627 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:22.186805 (+ 6178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:22.186809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:22.186836 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:22.187494 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.187510 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.189059 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.189067 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.191350 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.191363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.192944 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.192954 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.193154 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.193293 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 02:46:22.193368 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 02:46:22.193542 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":321,"threads_started":1}
I20260812 02:46:22.195214 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.178967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50956 (local address 127.17.196.195:42517)
0812 02:46:22.186139 (+ 7172us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.186148 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.186181 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:46:22.186282 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.186290 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.186576 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.187362 (+ 786us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:22.187763 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.188915 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.191514 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.192692 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.194385 (+ 1693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.194581 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.194657 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.194828 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6958,"thread_start_us":7777,"threads_started":1}
I20260812 02:46:22.195626 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.179721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38138 (local address 127.17.196.193:43881)
0812 02:46:22.185234 (+ 5513us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.185243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.185823 (+ 580us) server_negotiation.cc:407] Connection header received
0812 02:46:22.186140 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.186149 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.186382 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.186582 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:22.187908 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.189550 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.192610 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.194487 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.194696 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.194852 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.195119 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.195280 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5351,"thread_start_us":5172,"threads_started":1}
I20260812 02:46:22.196377 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.179186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38138)
0812 02:46:22.185704 (+ 6518us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:22.185724 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:22.185927 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:22.186852 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:22.186856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:22.186885 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:22.187677 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.187694 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.189721 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.189726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.192395 (+ 2669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.192415 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.194710 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.194718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.194894 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.194931 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:46:22.195893 (+ 962us) client_negotiation.cc:240] Negotiation successful
0812 02:46:22.196030 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6401,"mutex_wait_us":1281,"thread_start_us":1289,"threads_started":1}
I20260812 02:46:22.199106 1774 tablet_service.cc:1467] Processing CreateTablet for tablet 3366a061d2e849c2995bbb82492c246a (DEFAULT_TABLE table=test-table [id=46ace8106a4f43e4b987df481b7d19fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:22.199153 1932 tablet_service.cc:1467] Processing CreateTablet for tablet 3366a061d2e849c2995bbb82492c246a (DEFAULT_TABLE table=test-table [id=46ace8106a4f43e4b987df481b7d19fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:22.200403 1774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3366a061d2e849c2995bbb82492c246a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:22.200611 1932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3366a061d2e849c2995bbb82492c246a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:22.200027 1620 tablet_service.cc:1467] Processing CreateTablet for tablet 3366a061d2e849c2995bbb82492c246a (DEFAULT_TABLE table=test-table [id=46ace8106a4f43e4b987df481b7d19fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:22.201704 1620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3366a061d2e849c2995bbb82492c246a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:22.212472 2030 tablet_bootstrap.cc:492] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Bootstrap starting.
I20260812 02:46:22.215927 2030 tablet_bootstrap.cc:654] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:22.216902 2031 tablet_bootstrap.cc:492] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Bootstrap starting.
I20260812 02:46:22.217398 2030 log.cc:824] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:22.221670 2031 tablet_bootstrap.cc:654] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:22.222375 2032 tablet_bootstrap.cc:492] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Bootstrap starting.
I20260812 02:46:22.223661 2031 log.cc:824] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:22.226050 2032 tablet_bootstrap.cc:654] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:22.226665 2030 tablet_bootstrap.cc:492] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: No bootstrap required, opened a new log
I20260812 02:46:22.226963 2030 ts_tablet_manager.cc:1397] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.005s
I20260812 02:46:22.227294 2032 log.cc:824] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:22.228294 2031 tablet_bootstrap.cc:492] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: No bootstrap required, opened a new log
I20260812 02:46:22.228601 2031 ts_tablet_manager.cc:1397] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260812 02:46:22.230541 2032 tablet_bootstrap.cc:492] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: No bootstrap required, opened a new log
I20260812 02:46:22.230818 2032 ts_tablet_manager.cc:1397] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 02:46:22.238350 2032 raft_consensus.cc:348] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.238230 2030 raft_consensus.cc:348] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.238735 2030 raft_consensus.cc:374] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:22.238732 2032 raft_consensus.cc:374] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:22.238840 2030 raft_consensus.cc:729] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a092adcdb0434969b20f9350482e75b7, State: Initialized, Role: FOLLOWER
I20260812 02:46:22.238842 2032 raft_consensus.cc:729] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655222ec0c32432993a1b9dea125c5d1, State: Initialized, Role: FOLLOWER
I20260812 02:46:22.239356 2031 raft_consensus.cc:348] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.239527 2030 consensus_queue.cc:260] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.239787 2031 raft_consensus.cc:374] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:22.239547 2032 consensus_queue.cc:260] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.239879 2031 raft_consensus.cc:729] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df02ad79c3847af92272dfc3c64bad2, State: Initialized, Role: FOLLOWER
I20260812 02:46:22.240530 2031 consensus_queue.cc:260] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.248581 2030 ts_tablet_manager.cc:1428] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Time spent starting tablet: real 0.021s user 0.011s sys 0.010s
I20260812 02:46:22.249099 1847 heartbeater.cc:503] Master 127.17.196.254:37967 was elected leader, sending a full tablet report...
I20260812 02:46:22.259040 2010 heartbeater.cc:503] Master 127.17.196.254:37967 was elected leader, sending a full tablet report...
I20260812 02:46:22.259922 2031 ts_tablet_manager.cc:1428] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Time spent starting tablet: real 0.031s user 0.032s sys 0.000s
I20260812 02:46:22.260179 2032 ts_tablet_manager.cc:1428] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Time spent starting tablet: real 0.029s user 0.021s sys 0.008s
W20260812 02:46:22.264153 1848 tablet.cc:2369] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:46:22.348273 2011 tablet.cc:2369] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:22.438721 2037 raft_consensus.cc:482] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:46:22.439145 2037 raft_consensus.cc:504] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.441181 2037 leader_election.cc:288] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6df02ad79c3847af92272dfc3c64bad2 (127.17.196.195:42517), 655222ec0c32432993a1b9dea125c5d1 (127.17.196.193:43881)
I20260812 02:46:22.449345 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.441649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:51681 (local address 127.17.196.195:42517)
0812 02:46:22.441964 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.441975 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.442866 (+ 891us) server_negotiation.cc:407] Connection header received
0812 02:46:22.443130 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.443134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.443199 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.443317 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:22.444057 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.444911 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.447231 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.448108 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.448331 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.448435 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.448883 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.449046 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 02:46:22.449359 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.441924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:33621 (local address 127.17.196.193:43881)
0812 02:46:22.442242 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.442252 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.442933 (+ 681us) server_negotiation.cc:407] Connection header received
0812 02:46:22.443171 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.443175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.443253 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.443372 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:22.444145 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.444920 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.447257 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.448176 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.448404 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.448494 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.448919 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.449075 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 02:46:22.449664 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.441500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:42517 (local address 127.17.196.194:51681)
0812 02:46:22.442730 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:22.442748 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:22.442946 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:22.443354 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:22.443358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:22.443386 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:22.443886 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.443899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.445072 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.445080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.447046 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.447060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.448396 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.448405 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.448668 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.448717 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 02:46:22.448842 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 02:46:22.448995 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"mutex_wait_us":51,"thread_start_us":730,"threads_started":1}
I20260812 02:46:22.450093 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.441606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.17.196.194:33621)
0812 02:46:22.442799 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:22.442815 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:22.443004 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:22.443399 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:22.443405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:22.443434 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:22.443965 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.443976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.445089 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.445096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.447036 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:22.447053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.448455 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.448463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:22.448682 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.448762 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 02:46:22.448886 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 02:46:22.449036 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"mutex_wait_us":77,"thread_start_us":852,"threads_started":1}
I20260812 02:46:22.450951 1957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3366a061d2e849c2995bbb82492c246a" candidate_uuid: "a092adcdb0434969b20f9350482e75b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df02ad79c3847af92272dfc3c64bad2" is_pre_election: true
I20260812 02:46:22.451340 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3366a061d2e849c2995bbb82492c246a" candidate_uuid: "a092adcdb0434969b20f9350482e75b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655222ec0c32432993a1b9dea125c5d1" is_pre_election: true
I20260812 02:46:22.451529 1957 raft_consensus.cc:2454] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a092adcdb0434969b20f9350482e75b7 in term 0.
W20260812 02:46:22.451980 1696 tablet.cc:2369] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:22.451969 1645 raft_consensus.cc:2454] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a092adcdb0434969b20f9350482e75b7 in term 0.
I20260812 02:46:22.452303 1722 leader_election.cc:302] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df02ad79c3847af92272dfc3c64bad2, a092adcdb0434969b20f9350482e75b7; no voters:
I20260812 02:46:22.452803 2037 raft_consensus.cc:2781] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:46:22.452975 2037 raft_consensus.cc:482] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:46:22.453111 2037 raft_consensus.cc:3037] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:22.455484 2037 raft_consensus.cc:504] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.456163 2037 leader_election.cc:288] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [CANDIDATE]: Term 1 election: Requested vote from peers 6df02ad79c3847af92272dfc3c64bad2 (127.17.196.195:42517), 655222ec0c32432993a1b9dea125c5d1 (127.17.196.193:43881)
I20260812 02:46:22.456861 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3366a061d2e849c2995bbb82492c246a" candidate_uuid: "a092adcdb0434969b20f9350482e75b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655222ec0c32432993a1b9dea125c5d1"
I20260812 02:46:22.456925 1957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3366a061d2e849c2995bbb82492c246a" candidate_uuid: "a092adcdb0434969b20f9350482e75b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df02ad79c3847af92272dfc3c64bad2"
I20260812 02:46:22.457167 1957 raft_consensus.cc:3037] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:22.457154 1645 raft_consensus.cc:3037] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:22.459626 1957 raft_consensus.cc:2454] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a092adcdb0434969b20f9350482e75b7 in term 1.
I20260812 02:46:22.459900 1645 raft_consensus.cc:2454] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a092adcdb0434969b20f9350482e75b7 in term 1.
I20260812 02:46:22.460220 1722 leader_election.cc:302] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df02ad79c3847af92272dfc3c64bad2, a092adcdb0434969b20f9350482e75b7; no voters:
I20260812 02:46:22.460601 2037 raft_consensus.cc:2781] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:22.461153 2037 raft_consensus.cc:686] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 1 LEADER]: Becoming Leader. State: Replica: a092adcdb0434969b20f9350482e75b7, State: Running, Role: LEADER
I20260812 02:46:22.461737 2037 consensus_queue.cc:237] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:22.466840 1476 catalog_manager.cc:5495] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a092adcdb0434969b20f9350482e75b7 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "a092adcdb0434969b20f9350482e75b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } health_report { overall_health: UNKNOWN } } }
I20260812 02:46:22.494253 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.488201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.17.196.194:46121)
0812 02:46:22.488445 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.488453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.488482 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:46:22.488600 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.488604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.488688 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.488827 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:46:22.489308 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.490238 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.491690 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.492111 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.492208 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.492866 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:46:22.493434 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:46:22.493703 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.493793 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.493901 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:46:22.503132 1645 raft_consensus.cc:1261] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 1 FOLLOWER]: Refusing update from remote peer a092adcdb0434969b20f9350482e75b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:22.503131 1957 raft_consensus.cc:1261] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 1 FOLLOWER]: Refusing update from remote peer a092adcdb0434969b20f9350482e75b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:22.504220 2043 consensus_queue.cc:1035] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:22.504681 2037 consensus_queue.cc:1035] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:22.517223 2046 mvcc.cc:205] Tried to move back new op lower bound from 7317515397121753088 to 7317515396967211008. Current Snapshot: MvccSnapshot[applied={T|T < 7317515397121753088}]
I20260812 02:46:22.536605 1478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:46:22.536939 1478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:46:22.541375 1478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Sending DeleteTablet for 3 replicas of tablet 3366a061d2e849c2995bbb82492c246a
I20260812 02:46:22.542857 1620 tablet_service.cc:1514] Processing DeleteTablet for tablet 3366a061d2e849c2995bbb82492c246a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:22 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:38138
I20260812 02:46:22.543051 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 1549
I20260812 02:46:22.543256 1932 tablet_service.cc:1514] Processing DeleteTablet for tablet 3366a061d2e849c2995bbb82492c246a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:22 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:50956
I20260812 02:46:22.544051 1774 tablet_service.cc:1514] Processing DeleteTablet for tablet 3366a061d2e849c2995bbb82492c246a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:22 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:34030
I20260812 02:46:22.564141 2056 tablet_replica.cc:330] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: stopping tablet replica
I20260812 02:46:22.564395 2059 tablet_replica.cc:330] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: stopping tablet replica
I20260812 02:46:22.564735 2056 raft_consensus.cc:2229] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:46:22.564778 2059 raft_consensus.cc:2229] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:46:22.565330 2056 raft_consensus.cc:2258] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:22.565519 2059 raft_consensus.cc:2258] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:22.566383 2057 tablet_replica.cc:330] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: stopping tablet replica
I20260812 02:46:22.566830 2057 raft_consensus.cc:2229] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:46:22.567401 2057 raft_consensus.cc:2258] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:22.569868 2056 ts_tablet_manager.cc:1905] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:22.569868 2057 ts_tablet_manager.cc:1905] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:22.570083 2059 ts_tablet_manager.cc:1905] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:46:22 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:46:22.577570 2057 ts_tablet_manager.cc:1918] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:22.577764 2057 log.cc:1196] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/3366a061d2e849c2995bbb82492c246a
I20260812 02:46:22.578094 2059 ts_tablet_manager.cc:1918] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:22.578322 2057 ts_tablet_manager.cc:1939] T 3366a061d2e849c2995bbb82492c246a P 6df02ad79c3847af92272dfc3c64bad2: Deleting consensus metadata
I20260812 02:46:22.578373 2059 log.cc:1196] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/3366a061d2e849c2995bbb82492c246a
I20260812 02:46:22.579064 2059 ts_tablet_manager.cc:1939] T 3366a061d2e849c2995bbb82492c246a P a092adcdb0434969b20f9350482e75b7: Deleting consensus metadata
I20260812 02:46:22.581082 1461 catalog_manager.cc:4841] TS 6df02ad79c3847af92272dfc3c64bad2 (127.17.196.195:42517): tablet 3366a061d2e849c2995bbb82492c246a (table test-table [id=46ace8106a4f43e4b987df481b7d19fe]) successfully deleted
I20260812 02:46:22.582446 1461 catalog_manager.cc:4841] TS a092adcdb0434969b20f9350482e75b7 (127.17.196.194:46121): tablet 3366a061d2e849c2995bbb82492c246a (table test-table [id=46ace8106a4f43e4b987df481b7d19fe]) successfully deleted
I20260812 02:46:22.584460 2056 ts_tablet_manager.cc:1918] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:22.584645 2056 log.cc:1196] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/3366a061d2e849c2995bbb82492c246a
I20260812 02:46:22.585244 2056 ts_tablet_manager.cc:1939] T 3366a061d2e849c2995bbb82492c246a P 655222ec0c32432993a1b9dea125c5d1: Deleting consensus metadata
I20260812 02:46:22.586879 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.544928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.17.196.193:43881)
0812 02:46:22.545143 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.545150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.554988 (+ 9838us) server_negotiation.cc:407] Connection header received
0812 02:46:22.555649 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.555660 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.555806 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.555922 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:22.558786 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.560443 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.565834 (+ 5391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.566574 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.566714 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.578519 (+ 11805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:22.578544 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:22.578565 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:22.578624 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:22.582893 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.583965 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.583975 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.583978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.584167 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.584685 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.584694 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.584697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.585244 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:22.585624 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.586191 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.586457 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:46:22.587258 1462 catalog_manager.cc:4841] TS 655222ec0c32432993a1b9dea125c5d1 (127.17.196.193:43881): tablet 3366a061d2e849c2995bbb82492c246a (table test-table [id=46ace8106a4f43e4b987df481b7d19fe]) successfully deleted
I20260812 02:46:22.587713 1690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:22.728157 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 1549
I20260812 02:46:22.749722 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 1702
Aug 12 02:46:22 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:46:22.769872 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.750594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34052 (local address 127.17.196.194:46121)
0812 02:46:22.750774 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.750779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.750866 (+ 87us) server_negotiation.cc:407] Connection header received
0812 02:46:22.751056 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.751060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.751124 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.751211 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:22.753164 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.754189 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.756486 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.757132 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.757240 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.762662 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:22.762679 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:22.762697 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:22.762756 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:22.766384 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.767322 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.767330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.767333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.767505 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.767967 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.767974 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.767980 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.768380 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:22.768806 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.769157 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.769424 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:46:22.770701 1842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:22.890347 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 1702
I20260812 02:46:22.915771 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 1855
Aug 12 02:46:22 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:46:22.939989 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:22.917065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.17.196.195:42517)
0812 02:46:22.917286 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:22.917291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:22.917312 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:46:22.917820 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:22.917825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:22.917905 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:22.918006 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:22.919761 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.920483 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.922224 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:22.922659 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:22.922776 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:22.932892 (+ 10116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:22.932913 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:22.932931 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:22.932985 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:22.936474 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.937431 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.937440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.937443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.937784 (+ 341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:22.938260 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:22.938265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:22.938267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:22.938760 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:22.939042 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:22.939430 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 02:46:22.939686 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 02:46:22.940757 2002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:23.036142 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 1855
I20260812 02:46:23.057770 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 1440
I20260812 02:46:23.059016 1512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:23.173707 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 1440
I20260812 02:46:23.201043 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37967
--webserver_interface=127.17.196.254
--webserver_port=36989
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:37967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:23.565698 2064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:23.566092 2064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:23.566277 2064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:23.573854 2064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:23.574023 2064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:23.574136 2064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:23.574245 2064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:23.574332 2064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:23.584797 2064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:37967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=36989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:23.586289 2064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:23.587729 2064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:23.615552 2073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:23.615706 2072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:23.615700 2076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:23.616835 2064 server_base.cc:1034] running on GCE node
I20260812 02:46:23.618134 2064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:23.620018 2064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:23.621332 2064 hybrid_clock.cc:648] HybridClock initialized: now 1786502783621280 us; error 36 us; skew 500 ppm
I20260812 02:46:23.621886 2064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:23 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502783, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:23.626111 2064 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:46:23.628284 2064 webserver.cc:460] Webserver started at http://127.17.196.254:36989/ using document root <none> and password file <none>
I20260812 02:46:23.629161 2064 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:23.629350 2064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:23.636365 2064 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:46:23.639789 2085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:23.641266 2064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:46:23.641592 2064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "ad8407290538402c61b32f80c0bfc594"
server_key_iv: "320f8d174ca2a83ae61f208b43572059"
server_key_version: "encryptionkey@0"
I20260812 02:46:23.642763 2064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:23.681028 2064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:23.686751 2064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:23.687119 2064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:23.708412 2148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:37967 every 8 connection(s)
I20260812 02:46:23.708405 2064 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:37967
I20260812 02:46:23.710404 2064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:46:23.713325 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 2064
I20260812 02:46:23.714241 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:43881
--local_ip_for_outbound_sockets=127.17.196.193
--tserver_master_addrs=127.17.196.254:37967
--webserver_port=38289
--webserver_interface=127.17.196.193
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:46:23.718428 2150 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:46:23.722193 2150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Bootstrap starting.
I20260812 02:46:23.743820 2150 log.cc:824] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:23.757406 2150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:46:23.758080 2150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Bootstrap complete.
I20260812 02:46:23.768499 2150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:23.769526 2150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bb2a80fef614c0cbfdf7f0670da6ad7, State: Initialized, Role: FOLLOWER
I20260812 02:46:23.770309 2150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:23.770606 2150 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:46:23.770748 2150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:46:23.770942 2150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:46:23.775200 2150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:23.775799 2150 leader_election.cc:302] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bb2a80fef614c0cbfdf7f0670da6ad7; no voters:
I20260812 02:46:23.776531 2150 leader_election.cc:288] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 02:46:23.776684 2154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:46:23.778574 2154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 2 LEADER]: Becoming Leader. State: Replica: 2bb2a80fef614c0cbfdf7f0670da6ad7, State: Running, Role: LEADER
I20260812 02:46:23.779232 2154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:23.779508 2150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:23.783963 2156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb2a80fef614c0cbfdf7f0670da6ad7. Latest consensus state: current_term: 2 leader_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } } }
I20260812 02:46:23.783963 2155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } } }
I20260812 02:46:23.784512 2156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:23.784513 2155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:23.785588 2160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:46:23.802620 2160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:46:23.804189 2160 catalog_manager.cc:1259] Loaded cluster ID: a5969e16c29b48c797e14c2acf035473
I20260812 02:46:23.804307 2160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:46:23.807883 2160 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 02:46:23.826071 2160 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 02:46:23.836015 2160 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:46:23.838338 2160 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Loaded TSK: 0
I20260812 02:46:23.840164 2160 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:46:24.107581 2152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:24.107928 2152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:24.108110 2152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:24.115703 2152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:24.115871 2152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:24.116070 2152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:46:24.127562 2152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:43881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=38289
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:24.128993 2152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:24.130627 2152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:24.147418 2182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:24.148039 2185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:24.150002 2181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:24.151449 2152 server_base.cc:1034] running on GCE node
I20260812 02:46:24.152348 2152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:24.153607 2152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:24.154862 2152 hybrid_clock.cc:648] HybridClock initialized: now 1786502784154810 us; error 47 us; skew 500 ppm
I20260812 02:46:24.155238 2152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:24 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502784, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:24.159121 2152 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:46:24.161087 2152 webserver.cc:460] Webserver started at http://127.17.196.193:38289/ using document root <none> and password file <none>
I20260812 02:46:24.161988 2152 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:24.162200 2152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:24.169180 2152 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 02:46:24.172608 2193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:24.174326 2152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:46:24.174525 2152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "655222ec0c32432993a1b9dea125c5d1"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "39620fcec97e00bc8cad70b80f99177e"
server_key_iv: "9168d26243cc1647c7cdb8ae23812f48"
server_key_version: "encryptionkey@0"
I20260812 02:46:24.175661 2152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:24.198827 2152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:24.203745 2152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:24.204066 2152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:24.205150 2152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:24.206605 2152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:24.206732 2152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:24.206869 2152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:24.206964 2152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:24.243374 2152 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:43881
I20260812 02:46:24.243484 2330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:43881 every 8 connection(s)
I20260812 02:46:24.245150 2152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:46:24.247212 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 2152
I20260812 02:46:24.248085 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:46121
--local_ip_for_outbound_sockets=127.17.196.194
--tserver_master_addrs=127.17.196.254:37967
--webserver_port=46391
--webserver_interface=127.17.196.194
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:46:24 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502784, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:24.277238 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:24.249060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:59971)
0812 02:46:24.250154 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:24.250214 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:24.253283 (+ 3069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:24.255007 (+ 1724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:24.255022 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:24.255967 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:24.256860 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:24.256893 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:24.258797 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:24.258806 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:24.260936 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:24.260952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:24.262775 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:24.262781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:24.263039 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:24.263870 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:24.263898 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:24.266501 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:24.272142 (+ 5641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:24.272152 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:24.272166 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:24.272729 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:24.273369 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:24.273372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:24.273375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:24.273720 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:24.274815 (+ 1095us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:24.274821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:24.275167 (+ 346us) client_negotiation.cc:769] Sending connection context
0812 02:46:24.275436 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 02:46:24.276067 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":442,"threads_started":1}
I20260812 02:46:24.277916 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:24.248815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:59971 (local address 127.17.196.254:37967)
0812 02:46:24.249615 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:24.249639 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:24.250386 (+ 747us) server_negotiation.cc:407] Connection header received
0812 02:46:24.253955 (+ 3569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:24.253971 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:24.254406 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:24.254743 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:24.257172 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:24.258581 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:24.261131 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:24.262487 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:24.262737 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:24.266875 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:24.266890 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:24.266917 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:24.266986 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:24.271934 (+ 4948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:24.272914 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:24.272928 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:24.272941 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:24.273226 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:24.273849 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:24.273857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:24.273864 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:24.274649 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:24.274878 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:24.275799 (+ 921us) server_negotiation.cc:299] Negotiation successful
0812 02:46:24.276375 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":303,"threads_started":1}
I20260812 02:46:24.281690 2332 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37967
I20260812 02:46:24.282116 2332 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:24.283324 2332 heartbeater.cc:511] Master 127.17.196.254:37967 requested a full tablet report, sending...
I20260812 02:46:24.286588 2107 ts_manager.cc:194] Registered new tserver with Master: 655222ec0c32432993a1b9dea125c5d1 (127.17.196.193:43881)
I20260812 02:46:24.288201 2107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:59971
W20260812 02:46:24.617439 2337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:24.617789 2337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:24.617959 2337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:24.625363 2337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:24.625507 2337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:24.625742 2337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:46:24.636194 2337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:46121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=46391
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:24.637688 2337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:24.639006 2337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:24.652894 2346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:24.653039 2343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:24.653231 2344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:24.654269 2337 server_base.cc:1034] running on GCE node
I20260812 02:46:24.654909 2337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:24.655972 2337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:24.657225 2337 hybrid_clock.cc:648] HybridClock initialized: now 1786502784657185 us; error 45 us; skew 500 ppm
I20260812 02:46:24.657760 2337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:24 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502784, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:24.661758 2337 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:46:24.663561 2337 webserver.cc:460] Webserver started at http://127.17.196.194:46391/ using document root <none> and password file <none>
I20260812 02:46:24.664346 2337 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:24.664546 2337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:24.670835 2337 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:46:24.674336 2353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:24.675899 2337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 02:46:24.676115 2337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "a092adcdb0434969b20f9350482e75b7"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "c03f32e25e1c1cbefd0163dac1ac34de"
server_key_iv: "fde051860bb81ae434a5dcddae8812af"
server_key_version: "encryptionkey@0"
I20260812 02:46:24.677322 2337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:24.716353 2337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:24.721809 2337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:24.722108 2337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:24.723243 2337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:24.724686 2337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:24.724815 2337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:24.724956 2337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:24.725045 2337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:24.765066 2337 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:46121
I20260812 02:46:24.765131 2488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:46121 every 8 connection(s)
I20260812 02:46:24.766819 2337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:46:24.776670 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 2337
I20260812 02:46:24.777428 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:42517
--local_ip_for_outbound_sockets=127.17.196.195
--tserver_master_addrs=127.17.196.254:37967
--webserver_port=38391
--webserver_interface=127.17.196.195
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:46:24 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502784, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:24.792376 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:24.769947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:42379 (local address 127.17.196.254:37967)
0812 02:46:24.770187 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:24.770195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:24.771595 (+ 1400us) server_negotiation.cc:407] Connection header received
0812 02:46:24.773474 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:24.773479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:24.773647 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:24.773750 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:24.775383 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:24.776258 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:24.778257 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:24.779191 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:24.779381 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:24.784282 (+ 4901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:24.784293 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:24.784302 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:24.784374 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:24.788445 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:24.789335 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:24.789344 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:24.789350 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:24.789582 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:24.790186 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:24.790190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:24.790192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:24.790502 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:24.790619 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:24.791887 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0812 02:46:24.792106 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 02:46:24.794134 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:24.770585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:42379)
0812 02:46:24.771408 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:24.771453 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:24.773167 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:24.773987 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:24.774004 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:24.774561 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:24.775202 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:24.775220 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:24.776461 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:24.776465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:24.778043 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:24.778056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:24.779543 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:24.779555 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:24.779878 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:24.780956 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:24.780998 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:24.783999 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:24.788623 (+ 4624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:24.788631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:24.788647 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:24.789170 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:24.789720 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:24.789727 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:24.789733 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:24.790051 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:24.790663 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:24.790675 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:24.791290 (+ 615us) client_negotiation.cc:769] Sending connection context
0812 02:46:24.791663 (+ 373us) client_negotiation.cc:240] Negotiation successful
0812 02:46:24.792360 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":336,"threads_started":1}
I20260812 02:46:24.795917 2489 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37967
I20260812 02:46:24.796281 2489 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:24.797104 2489 heartbeater.cc:511] Master 127.17.196.254:37967 requested a full tablet report, sending...
I20260812 02:46:24.798822 2107 ts_manager.cc:194] Registered new tserver with Master: a092adcdb0434969b20f9350482e75b7 (127.17.196.194:46121)
I20260812 02:46:24.799692 2107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:42379
W20260812 02:46:25.140451 2492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:25.140827 2492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:25.140992 2492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:25.148748 2492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:25.148918 2492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:25.149103 2492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:46:25.159330 2492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:42517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=38391
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:25.160837 2492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:25.162286 2492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:25.176469 2499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:25.176445 2500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:25.176398 2502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:25.177417 2492 server_base.cc:1034] running on GCE node
I20260812 02:46:25.178159 2492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:25.179293 2492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:25.180536 2492 hybrid_clock.cc:648] HybridClock initialized: now 1786502785180459 us; error 72 us; skew 500 ppm
I20260812 02:46:25.180997 2492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:25 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502785, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:25.184537 2492 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:46:25.186249 2492 webserver.cc:460] Webserver started at http://127.17.196.195:38391/ using document root <none> and password file <none>
I20260812 02:46:25.187000 2492 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:25.187183 2492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:25.193217 2492 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 02:46:25.196508 2509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:25.198238 2492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:46:25.198439 2492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "6df02ad79c3847af92272dfc3c64bad2"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "da3320dd20b745ae73a95dafdf5b71e6"
server_key_iv: "211cd0c66d163a8742a2594bdda28ee5"
server_key_version: "encryptionkey@0"
I20260812 02:46:25.199532 2492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:25.217700 2492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:25.223289 2492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:25.223596 2492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:25.224752 2492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:25.226272 2492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:25.226410 2492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:25.226540 2492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:25.226634 2492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:25.264482 2492 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:42517
I20260812 02:46:25.264621 2641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:42517 every 8 connection(s)
I20260812 02:46:25.266140 2492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:46:25.274374 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 2492
Aug 12 02:46:25 dist-test-slave-ss4n krb5kdc[1427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502785, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:25.291800 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.268921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:57137 (local address 127.17.196.254:37967)
0812 02:46:25.269148 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.269155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.270462 (+ 1307us) server_negotiation.cc:407] Connection header received
0812 02:46:25.272302 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.272307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.272378 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.272521 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:25.274253 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.275166 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.277780 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.278544 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.278690 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.283579 (+ 4889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:25.283588 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:25.283596 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:25.283647 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:25.287263 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:25.288512 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:25.288517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:25.288519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:25.288740 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:25.289424 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:25.289428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:25.289430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:25.289852 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:25.289986 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.291148 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.291431 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:46:25.292393 2332 heartbeater.cc:503] Master 127.17.196.254:37967 was elected leader, sending a full tablet report...
I20260812 02:46:25.293035 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.269462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:57137)
0812 02:46:25.270282 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:25.270321 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:25.272023 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:25.272704 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:25.272712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:25.273271 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:25.274049 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.274077 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.275383 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.275389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.277537 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.277556 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.278790 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.278795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.279018 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.279965 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:25.279991 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:25.283286 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:25.287474 (+ 4188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:25.287489 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:25.287514 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:25.288348 (+ 834us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:25.288912 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:25.288918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:25.288925 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:25.289271 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:25.290004 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:25.290013 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:25.290584 (+ 571us) client_negotiation.cc:769] Sending connection context
0812 02:46:25.290886 (+ 302us) client_negotiation.cc:240] Negotiation successful
0812 02:46:25.291342 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":280,"threads_started":1}
I20260812 02:46:25.294843 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.275378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.17.196.254:37967)
0812 02:46:25.275998 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.276003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.276307 (+ 304us) server_negotiation.cc:407] Connection header received
0812 02:46:25.276547 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.276571 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.276660 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.276791 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:25.279044 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.280195 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.282165 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.282879 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.283042 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.286825 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:25.286831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:25.286834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:25.286889 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:25.290409 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:25.291693 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:25.291698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:25.291702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:25.291856 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:25.292594 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:25.292603 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:25.292606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:25.293093 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:25.293446 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.294151 (+ 705us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.294406 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":340,"threads_started":1}
I20260812 02:46:25.295252 2642 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37967
I20260812 02:46:25.295710 2642 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:25.296772 2642 heartbeater.cc:511] Master 127.17.196.254:37967 requested a full tablet report, sending...
I20260812 02:46:25.298583 2107 ts_manager.cc:194] Registered new tserver with Master: 6df02ad79c3847af92272dfc3c64bad2 (127.17.196.195:42517)
I20260812 02:46:25.299451 2107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:57137
I20260812 02:46:25.311213 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:46:25.319839 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.312384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.17.196.254:37967)
0812 02:46:25.312717 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.312722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.312892 (+ 170us) server_negotiation.cc:407] Connection header received
0812 02:46:25.313114 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.313118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.313275 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.313414 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:46:25.314014 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.314761 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.316841 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.317360 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.317470 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.318231 (+ 761us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:46:25.318977 (+ 746us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:46:25.319115 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.319373 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.319522 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 02:46:25.321230 2107 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:46:25.328979 2107 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:46:25.355057 2656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.342967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:46121 (local address 127.0.0.1:34066)
0812 02:46:25.344803 (+ 1836us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:25.344848 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:25.345147 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:25.346423 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:25.346435 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:25.346482 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:25.347142 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.347157 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.349571 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.349578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.351718 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.351737 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.353987 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.353995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.354223 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.354293 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 02:46:25.354437 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 02:46:25.354637 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1540,"thread_start_us":1371,"threads_started":1}
I20260812 02:46:25.355083 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.343249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.17.196.194:46121)
0812 02:46:25.343965 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.343974 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.344970 (+ 996us) server_negotiation.cc:407] Connection header received
0812 02:46:25.345422 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.345433 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.345788 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.345985 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:25.347364 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.349351 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.351908 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.353739 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.353921 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.354146 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.354510 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.354723 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":395,"threads_started":1}
I20260812 02:46:25.357652 2654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.343669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38164)
0812 02:46:25.346514 (+ 2845us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:25.346558 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:25.346719 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:25.349423 (+ 2704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:25.349428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:25.349456 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:25.349889 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.349900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.352034 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.352039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.354473 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.354489 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.356640 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.356648 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.356860 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.356900 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:46:25.356981 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 02:46:25.357142 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2668,"mutex_wait_us":30,"thread_start_us":808,"threads_started":1}
I20260812 02:46:25.358178 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.346027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.17.196.195:42517)
0812 02:46:25.347574 (+ 1547us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.347587 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.348660 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 02:46:25.348824 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.348833 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.349110 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.349326 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:25.350525 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.352351 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.354884 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.356474 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.356691 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.356880 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.357629 (+ 749us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.357809 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1306,"thread_start_us":1050,"threads_started":1}
I20260812 02:46:25.358544 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.344374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38164 (local address 127.17.196.193:43881)
0812 02:46:25.348214 (+ 3840us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.348225 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.348824 (+ 599us) server_negotiation.cc:407] Connection header received
0812 02:46:25.348935 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.348943 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.349201 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.350092 (+ 891us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:25.350282 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.351879 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.354707 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.356330 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.357743 (+ 1413us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.357911 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.358000 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.358199 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3577,"thread_start_us":3411,"threads_started":1}
I20260812 02:46:25.358937 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.345344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:42517 (local address 127.0.0.1:50962)
0812 02:46:25.348508 (+ 3164us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:25.348527 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:25.348666 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:25.349701 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:25.349708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:25.349749 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:25.350306 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.350318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.352526 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.352542 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.354691 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.354704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.357053 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.357060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.357278 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.357317 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:46:25.357391 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 02:46:25.358555 (+ 1164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3047,"mutex_wait_us":33,"thread_start_us":2903,"threads_started":1}
I20260812 02:46:25.360165 2412 tablet_service.cc:1467] Processing CreateTablet for tablet 0f107f6b05584355bc6609d01f13ef21 (DEFAULT_TABLE table=test-table [id=7a5d2a555f7d42dc9ebffd226aae7b01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:25.362035 2412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f107f6b05584355bc6609d01f13ef21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:25.362491 2566 tablet_service.cc:1467] Processing CreateTablet for tablet 0f107f6b05584355bc6609d01f13ef21 (DEFAULT_TABLE table=test-table [id=7a5d2a555f7d42dc9ebffd226aae7b01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:25.362705 2251 tablet_service.cc:1467] Processing CreateTablet for tablet 0f107f6b05584355bc6609d01f13ef21 (DEFAULT_TABLE table=test-table [id=7a5d2a555f7d42dc9ebffd226aae7b01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:25.363893 2566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f107f6b05584355bc6609d01f13ef21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:25.364245 2251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f107f6b05584355bc6609d01f13ef21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:25.377660 2662 tablet_bootstrap.cc:492] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Bootstrap starting.
I20260812 02:46:25.380434 2665 tablet_bootstrap.cc:492] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Bootstrap starting.
I20260812 02:46:25.382441 2662 tablet_bootstrap.cc:654] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:25.382526 2666 tablet_bootstrap.cc:492] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Bootstrap starting.
I20260812 02:46:25.383955 2662 log.cc:824] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:25.384274 2665 tablet_bootstrap.cc:654] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:25.385746 2665 log.cc:824] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:25.386349 2666 tablet_bootstrap.cc:654] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:25.387710 2666 log.cc:824] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:25.388633 2662 tablet_bootstrap.cc:492] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: No bootstrap required, opened a new log
I20260812 02:46:25.388942 2662 ts_tablet_manager.cc:1397] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260812 02:46:25.389101 2665 tablet_bootstrap.cc:492] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: No bootstrap required, opened a new log
I20260812 02:46:25.389396 2665 ts_tablet_manager.cc:1397] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 02:46:25.390995 2666 tablet_bootstrap.cc:492] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: No bootstrap required, opened a new log
I20260812 02:46:25.391263 2666 ts_tablet_manager.cc:1397] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 02:46:25.398034 2665 raft_consensus.cc:348] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.398483 2665 raft_consensus.cc:374] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:25.398603 2665 raft_consensus.cc:729] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a092adcdb0434969b20f9350482e75b7, State: Initialized, Role: FOLLOWER
I20260812 02:46:25.399326 2665 consensus_queue.cc:260] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.400022 2662 raft_consensus.cc:348] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.400501 2662 raft_consensus.cc:374] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:25.400597 2662 raft_consensus.cc:729] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df02ad79c3847af92272dfc3c64bad2, State: Initialized, Role: FOLLOWER
I20260812 02:46:25.400707 2489 heartbeater.cc:503] Master 127.17.196.254:37967 was elected leader, sending a full tablet report...
I20260812 02:46:25.401460 2665 ts_tablet_manager.cc:1428] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260812 02:46:25.401322 2662 consensus_queue.cc:260] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.402630 2642 heartbeater.cc:503] Master 127.17.196.254:37967 was elected leader, sending a full tablet report...
I20260812 02:46:25.402263 2666 raft_consensus.cc:348] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.403116 2666 raft_consensus.cc:374] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:25.403218 2666 raft_consensus.cc:729] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655222ec0c32432993a1b9dea125c5d1, State: Initialized, Role: FOLLOWER
I20260812 02:46:25.403304 2662 ts_tablet_manager.cc:1428] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260812 02:46:25.403800 2666 consensus_queue.cc:260] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.406754 2666 ts_tablet_manager.cc:1428] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260812 02:46:25.498148 2333 tablet.cc:2369] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:46:25.517936 2643 tablet.cc:2369] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:46:25.519068 2490 tablet.cc:2369] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:25.682332 2673 raft_consensus.cc:482] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:46:25.682890 2673 raft_consensus.cc:504] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.684957 2673 leader_election.cc:288] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a092adcdb0434969b20f9350482e75b7 (127.17.196.194:46121), 655222ec0c32432993a1b9dea125c5d1 (127.17.196.193:43881)
I20260812 02:46:25.692091 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.685659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:58633 (local address 127.17.196.194:46121)
0812 02:46:25.685860 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.685868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.685893 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:46:25.685995 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.686004 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.686099 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.686223 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:25.686660 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.687650 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.689596 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.691121 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.691278 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.691351 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.691745 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.691854 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:46:25.692103 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.685245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:46121 (local address 127.17.196.195:58633)
0812 02:46:25.685611 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:25.685637 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:25.685793 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:25.686255 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:25.686259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:25.686282 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:25.686478 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.686489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.687787 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.687793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.689375 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.689390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.691312 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.691319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.691520 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.691570 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 02:46:25.691675 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 02:46:25.691816 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 02:46:25.693239 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.685632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:50707 (local address 127.17.196.193:43881)
0812 02:46:25.685836 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.685843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.686471 (+ 628us) server_negotiation.cc:407] Connection header received
0812 02:46:25.686627 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.686640 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.686710 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.686778 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:25.687359 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.688391 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.691136 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.692358 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.692528 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.692601 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.692943 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.693048 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:46:25.693542 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.685245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.17.196.195:50707)
0812 02:46:25.686348 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:25.686367 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:25.686495 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:25.686823 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:25.686827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:25.686851 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:25.687202 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.687214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.688591 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.688596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.690913 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:25.690934 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.692620 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.692626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:25.692746 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.692770 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:46:25.692900 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 02:46:25.693049 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":265,"thread_start_us":497,"threads_started":1}
I20260812 02:46:25.693399 2435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f107f6b05584355bc6609d01f13ef21" candidate_uuid: "6df02ad79c3847af92272dfc3c64bad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a092adcdb0434969b20f9350482e75b7" is_pre_election: true
I20260812 02:46:25.694015 2435 raft_consensus.cc:2454] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6df02ad79c3847af92272dfc3c64bad2 in term 0.
I20260812 02:46:25.694774 2514 leader_election.cc:302] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df02ad79c3847af92272dfc3c64bad2, a092adcdb0434969b20f9350482e75b7; no voters:
I20260812 02:46:25.695003 2277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f107f6b05584355bc6609d01f13ef21" candidate_uuid: "6df02ad79c3847af92272dfc3c64bad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655222ec0c32432993a1b9dea125c5d1" is_pre_election: true
I20260812 02:46:25.695261 2673 raft_consensus.cc:2781] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:46:25.695439 2673 raft_consensus.cc:482] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:46:25.695603 2673 raft_consensus.cc:3037] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:25.695614 2277 raft_consensus.cc:2454] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6df02ad79c3847af92272dfc3c64bad2 in term 0.
I20260812 02:46:25.698042 2673 raft_consensus.cc:504] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.698599 2673 leader_election.cc:288] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [CANDIDATE]: Term 1 election: Requested vote from peers a092adcdb0434969b20f9350482e75b7 (127.17.196.194:46121), 655222ec0c32432993a1b9dea125c5d1 (127.17.196.193:43881)
I20260812 02:46:25.699134 2277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f107f6b05584355bc6609d01f13ef21" candidate_uuid: "6df02ad79c3847af92272dfc3c64bad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655222ec0c32432993a1b9dea125c5d1"
I20260812 02:46:25.699141 2435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f107f6b05584355bc6609d01f13ef21" candidate_uuid: "6df02ad79c3847af92272dfc3c64bad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a092adcdb0434969b20f9350482e75b7"
I20260812 02:46:25.699339 2277 raft_consensus.cc:3037] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:25.699349 2435 raft_consensus.cc:3037] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:25.701646 2277 raft_consensus.cc:2454] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6df02ad79c3847af92272dfc3c64bad2 in term 1.
I20260812 02:46:25.701666 2435 raft_consensus.cc:2454] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6df02ad79c3847af92272dfc3c64bad2 in term 1.
I20260812 02:46:25.702099 2515 leader_election.cc:302] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655222ec0c32432993a1b9dea125c5d1, 6df02ad79c3847af92272dfc3c64bad2; no voters:
I20260812 02:46:25.702489 2673 raft_consensus.cc:2781] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:25.702963 2673 raft_consensus.cc:686] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 1 LEADER]: Becoming Leader. State: Replica: 6df02ad79c3847af92272dfc3c64bad2, State: Running, Role: LEADER
I20260812 02:46:25.703486 2673 consensus_queue.cc:237] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } }
I20260812 02:46:25.708393 2105 catalog_manager.cc:5495] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6df02ad79c3847af92272dfc3c64bad2 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "6df02ad79c3847af92272dfc3c64bad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6df02ad79c3847af92272dfc3c64bad2" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 42517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 } health_report { overall_health: UNKNOWN } } }
I20260812 02:46:25.727545 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.721813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.17.196.195:42517)
0812 02:46:25.722059 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.722067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.722122 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:46:25.722420 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.722424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.722505 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.722667 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:46:25.723088 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.723838 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.725249 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.725728 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.725865 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.726407 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:46:25.726949 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:46:25.727129 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.727186 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.727274 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:46:25.736595 2435 raft_consensus.cc:1261] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 1 FOLLOWER]: Refusing update from remote peer 6df02ad79c3847af92272dfc3c64bad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:25.736613 2277 raft_consensus.cc:1261] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 1 FOLLOWER]: Refusing update from remote peer 6df02ad79c3847af92272dfc3c64bad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:25.737608 2677 consensus_queue.cc:1035] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a092adcdb0434969b20f9350482e75b7" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 46121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:25.738003 2673 consensus_queue.cc:1035] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "655222ec0c32432993a1b9dea125c5d1" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 43881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:25.753681 2680 mvcc.cc:205] Tried to move back new op lower bound from 7317515410366386176 to 7317515410245840896. Current Snapshot: MvccSnapshot[applied={T|T < 7317515410366386176}]
I20260812 02:46:25.769611 2107 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:46:25.769948 2107 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:46:25.774024 2107 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Sending DeleteTablet for 3 replicas of tablet 0f107f6b05584355bc6609d01f13ef21
I20260812 02:46:25.775430 2251 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f107f6b05584355bc6609d01f13ef21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:25 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:38164
I20260812 02:46:25.775449 2412 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f107f6b05584355bc6609d01f13ef21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:25 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:34066
I20260812 02:46:25.776618 2566 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f107f6b05584355bc6609d01f13ef21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:25 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:50962
I20260812 02:46:25.784771 2697 tablet_replica.cc:330] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: stopping tablet replica
I20260812 02:46:25.785842 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 2152
I20260812 02:46:25.787963 2697 raft_consensus.cc:2229] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:46:25.789310 2697 raft_consensus.cc:2258] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:25.795769 2697 ts_tablet_manager.cc:1905] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:25.800379 2696 tablet_replica.cc:330] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: stopping tablet replica
I20260812 02:46:25.801002 2696 raft_consensus.cc:2229] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:46:25.801811 2696 raft_consensus.cc:2258] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:25.802888 2695 tablet_replica.cc:330] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: stopping tablet replica
I20260812 02:46:25.803671 2695 raft_consensus.cc:2229] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:46:25.804208 2695 raft_consensus.cc:2258] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:25.806447 2696 ts_tablet_manager.cc:1905] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:25.809304 2697 ts_tablet_manager.cc:1918] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:25.809636 2697 log.cc:1196] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/0f107f6b05584355bc6609d01f13ef21
I20260812 02:46:25.809950 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.786817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.17.196.193:43881)
0812 02:46:25.787016 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.787023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.787193 (+ 170us) server_negotiation.cc:407] Connection header received
0812 02:46:25.787447 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.787463 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.787583 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.787702 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:25.789880 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.791687 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.794180 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.794944 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.795186 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.799471 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:25.799500 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:25.799522 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:25.799588 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:25.804414 (+ 4826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:25.805711 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:25.805722 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:25.805727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:25.805937 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:25.806516 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:25.806522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:25.806525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:25.807046 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:25.807417 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.809151 (+ 1734us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.809462 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:46:25.811018 2324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:25.811062 2697 ts_tablet_manager.cc:1939] T 0f107f6b05584355bc6609d01f13ef21 P 6df02ad79c3847af92272dfc3c64bad2: Deleting consensus metadata
I20260812 02:46:25.813211 2088 catalog_manager.cc:4841] TS 6df02ad79c3847af92272dfc3c64bad2 (127.17.196.195:42517): tablet 0f107f6b05584355bc6609d01f13ef21 (table test-table [id=7a5d2a555f7d42dc9ebffd226aae7b01]) successfully deleted
I20260812 02:46:25.813382 2695 ts_tablet_manager.cc:1905] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:25.819774 2695 ts_tablet_manager.cc:1918] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:25.820089 2695 log.cc:1196] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/0f107f6b05584355bc6609d01f13ef21
I20260812 02:46:25.820849 2695 ts_tablet_manager.cc:1939] T 0f107f6b05584355bc6609d01f13ef21 P a092adcdb0434969b20f9350482e75b7: Deleting consensus metadata
I20260812 02:46:25.822698 2088 catalog_manager.cc:4841] TS a092adcdb0434969b20f9350482e75b7 (127.17.196.194:46121): tablet 0f107f6b05584355bc6609d01f13ef21 (table test-table [id=7a5d2a555f7d42dc9ebffd226aae7b01]) successfully deleted
I20260812 02:46:25.926069 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 2152
I20260812 02:46:25.951685 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 2337
W20260812 02:46:25.951686 2089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.196.193:43881 (error 108)
I20260812 02:46:25.958206 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.953707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38180)
0812 02:46:25.957798 (+ 4091us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:25.957980 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3910,"spinlock_wait_cycles":19328,"thread_start_us":3886,"threads_started":1}
W20260812 02:46:25.958566 2089 catalog_manager.cc:4584] TS 655222ec0c32432993a1b9dea125c5d1 (127.17.196.193:43881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f107f6b05584355bc6609d01f13ef21: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
I20260812 02:46:25.971051 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:25.952729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.17.196.194:46121)
0812 02:46:25.952950 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:25.952971 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:25.953056 (+ 85us) server_negotiation.cc:407] Connection header received
0812 02:46:25.953233 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:25.953239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:25.953337 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:25.953554 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:25.956053 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.957184 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.959179 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:25.959939 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:25.960100 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:25.963235 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:25.963255 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:25.963278 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:25.963336 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:25.967488 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:25.968334 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:25.968345 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:25.968349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:25.968535 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:25.968986 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:25.968991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:25.968995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:25.969567 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:25.969890 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:25.970354 (+ 464us) server_negotiation.cc:299] Negotiation successful
0812 02:46:25.970615 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:46:25.971946 2482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:26.012418 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:26.011804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38186)
0812 02:46:26.012078 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:26.012226 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 02:46:26.014034 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:26.013302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38194)
0812 02:46:26.013693 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:26.013807 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 02:46:26.075125 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:26.074483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38198)
0812 02:46:26.074796 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:26.074914 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260812 02:46:26.076400 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 2337
I20260812 02:46:26.079172 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:26.078558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38208)
0812 02:46:26.078855 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:26.078961 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260812 02:46:26.109282 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 2492
I20260812 02:46:26.130457 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:26.110520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.17.196.195:42517)
0812 02:46:26.110806 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:26.110813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:26.110839 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:46:26.110914 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:26.110920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:26.111036 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:26.111167 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:26.112544 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:26.113956 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:26.115401 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:26.116171 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:26.116361 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:26.119388 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:26.119406 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:26.119429 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:26.119514 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:26.124905 (+ 5391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:26.126043 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:26.126057 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:26.126062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:26.126326 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:26.127150 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:26.127157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:26.127178 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:26.127856 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:26.128215 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:26.129711 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0812 02:46:26.129915 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260812 02:46:26.131291 2634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:26.162106 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:26.161471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38220)
0812 02:46:26.161813 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:26.161914 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260812 02:46:26.163359 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:26.162884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38228)
0812 02:46:26.163109 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:26.163192 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260812 02:46:26.259218 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 2492
I20260812 02:46:26.285744 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 2064
I20260812 02:46:26.287262 2144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:26.423756 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:26.423148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:43881 (local address 127.0.0.1:38238)
0812 02:46:26.423429 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:26.423561 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:43881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 02:46:26.423938 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 2064
I20260812 02:46:26.452109 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37967
--webserver_interface=127.17.196.254
--webserver_port=36989
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:37967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:26.824963 2705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:26.825296 2705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:26.825392 2705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:26.833387 2705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:26.833606 2705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:26.833680 2705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:26.833746 2705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:26.833792 2705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:26.844509 2705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:37967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=36989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:26.845961 2705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:26.847266 2705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:26.860222 2714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:26.860199 2717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:26.860114 2715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:26.861142 2705 server_base.cc:1034] running on GCE node
I20260812 02:46:26.862105 2705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:26.863692 2705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:26.864936 2705 hybrid_clock.cc:648] HybridClock initialized: now 1786502786864884 us; error 47 us; skew 500 ppm
I20260812 02:46:26.865314 2705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:26 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502786, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:26.868872 2705 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:46:26.870563 2705 webserver.cc:460] Webserver started at http://127.17.196.254:36989/ using document root <none> and password file <none>
I20260812 02:46:26.871285 2705 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:26.871433 2705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:26.877202 2705 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:46:26.882790 2724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:26.884291 2705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:46:26.884534 2705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "ad8407290538402c61b32f80c0bfc594"
server_key_iv: "320f8d174ca2a83ae61f208b43572059"
server_key_version: "encryptionkey@0"
I20260812 02:46:26.885695 2705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:26.916100 2705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:26.921121 2705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:26.921432 2705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:26.942396 2705 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:37967
I20260812 02:46:26.942417 2786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:37967 every 8 connection(s)
I20260812 02:46:26.944124 2705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:46:26.951428 2787 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:46:26.953522 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 2705
I20260812 02:46:26.954572 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:43881
--local_ip_for_outbound_sockets=127.17.196.193
--tserver_master_addrs=127.17.196.254:37967
--webserver_port=38289
--webserver_interface=127.17.196.193
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:46:26.955816 2787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Bootstrap starting.
I20260812 02:46:26.980311 2787 log.cc:824] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:26.998420 2787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:46:26.998992 2787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7: Bootstrap complete.
I20260812 02:46:27.008404 2787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:27.009398 2787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bb2a80fef614c0cbfdf7f0670da6ad7, State: Initialized, Role: FOLLOWER
I20260812 02:46:27.010171 2787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:27.010408 2787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:46:27.010558 2787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:46:27.010764 2787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 2 FOLLOWER]: Advancing to term 3
I20260812 02:46:27.014904 2787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:27.015419 2787 leader_election.cc:302] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bb2a80fef614c0cbfdf7f0670da6ad7; no voters:
I20260812 02:46:27.016155 2787 leader_election.cc:288] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 02:46:27.016304 2791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 02:46:27.017917 2791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [term 3 LEADER]: Becoming Leader. State: Replica: 2bb2a80fef614c0cbfdf7f0670da6ad7, State: Running, Role: LEADER
I20260812 02:46:27.018499 2791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } }
I20260812 02:46:27.018724 2787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:27.021374 2793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } } }
I20260812 02:46:27.021878 2793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:27.023635 2794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb2a80fef614c0cbfdf7f0670da6ad7. Latest consensus state: current_term: 3 leader_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb2a80fef614c0cbfdf7f0670da6ad7" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37967 } } }
I20260812 02:46:27.024010 2794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb2a80fef614c0cbfdf7f0670da6ad7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:27.024345 2797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:46:27.034196 2797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:46:27.036028 2797 catalog_manager.cc:1259] Loaded cluster ID: a5969e16c29b48c797e14c2acf035473
I20260812 02:46:27.036211 2797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 02:46:27.042883 2797 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc12979e9ad google::LogMessage::Fail() at ??:0
@ 0x7fc1297a188a google::LogMessage::SendToLog() at ??:0
@ 0x7fc12979e4de google::LogMessage::Flush() at ??:0
@ 0x7fc12979fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc13f1d1512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc13f1ce976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc13f256610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc13f256439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc12c3ea345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc12c3ed91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc12c3ed799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc12c3c8eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc12f6a26db start_thread at ??:0
@ 0x7fc12863861f clone at ??:0
W20260812 02:46:27.360198 2789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:27.360579 2789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:27.360752 2789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:27.368582 2789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:27.368759 2789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:27.368959 2789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:46:27.379266 2789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:43881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=38289
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:27.380724 2789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:27.382187 2789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:27.396155 2839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:27.396128 2841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:27.396400 2838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:27.397181 2789 server_base.cc:1034] running on GCE node
I20260812 02:46:27.398011 2789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:27.399081 2789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:27.400354 2789 hybrid_clock.cc:648] HybridClock initialized: now 1786502787400296 us; error 51 us; skew 500 ppm
I20260812 02:46:27.400724 2789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:27 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502787, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:27.404399 2789 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:46:27.406222 2789 webserver.cc:460] Webserver started at http://127.17.196.193:38289/ using document root <none> and password file <none>
I20260812 02:46:27.406983 2789 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:27.407136 2789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:27.412904 2789 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:46:27.416105 2848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:27.417667 2789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:46:27.417856 2789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "655222ec0c32432993a1b9dea125c5d1"
format_stamp: "Formatted at 2026-08-12 02:46:20 on dist-test-slave-ss4n"
server_key: "39620fcec97e00bc8cad70b80f99177e"
server_key_iv: "9168d26243cc1647c7cdb8ae23812f48"
server_key_version: "encryptionkey@0"
I20260812 02:46:27.418872 2789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:27.441067 2789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:27.446425 2789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:27.446847 2789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:27.448061 2789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:27.450449 2855 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 02:46:27.456593 2855 ts_tablet_manager.cc:1852] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 02:46:27.457782 2855 ts_tablet_manager.cc:1905] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:27.465744 2855 ts_tablet_manager.cc:1918] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:27.466032 2855 log.cc:1196] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/0f107f6b05584355bc6609d01f13ef21
I20260812 02:46:27.466555 2855 ts_tablet_manager.cc:1939] T 0f107f6b05584355bc6609d01f13ef21 P 655222ec0c32432993a1b9dea125c5d1: Deleting consensus metadata
I20260812 02:46:27.467020 2789 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 02:46:27.467185 2789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.001s sys 0.000s
I20260812 02:46:27.467338 2789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:27.467487 2789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:27.504829 2789 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:43881
I20260812 02:46:27.504928 2979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:43881 every 8 connection(s)
I20260812 02:46:27.506233 2789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:46:27.511510 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:27.509335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:41913)
0812 02:46:27.510277 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:27.510553 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":405,"threads_started":1}
W20260812 02:46:27.512382 2981 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:37967 (0 consecutive failures): Network error: Failed to ping master at 127.17.196.254:37967: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
I20260812 02:46:27.512548 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 2789
I20260812 02:46:27.513403 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:46121
--local_ip_for_outbound_sockets=127.17.196.194
--tserver_master_addrs=127.17.196.254:37967
--webserver_port=46391
--webserver_interface=127.17.196.194
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:46:27.513463 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:27.513096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:57449)
0812 02:46:27.513265 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:27.513341 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 02:46:27.514477 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:27.514090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:53093)
0812 02:46:27.514251 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:27.514340 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 02:46:27.514742 2981 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:46:27.884146 2984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:27.884464 2984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:27.884570 2984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:27.892556 2984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:27.892685 2984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:27.892800 2984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:46:27.903193 2984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:46121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=46391
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:27.904608 2984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:27.906029 2984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:27.923552 2992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:27.924002 2990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:27.923664 2995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:27.926991 2984 server_base.cc:1034] running on GCE node
I20260812 02:46:27.927806 2984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:27.928974 2984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:27.930217 2984 hybrid_clock.cc:648] HybridClock initialized: now 1786502787930154 us; error 45 us; skew 500 ppm
I20260812 02:46:27.930626 2984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:27 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502787, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:27.939980 2984 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:46:27.941939 2984 webserver.cc:460] Webserver started at http://127.17.196.194:46391/ using document root <none> and password file <none>
I20260812 02:46:27.942795 2984 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:27.942950 2984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:27.949139 2984 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 02:46:27.952445 3002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:27.954219 2984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:46:27.954427 2984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "a092adcdb0434969b20f9350482e75b7"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "c03f32e25e1c1cbefd0163dac1ac34de"
server_key_iv: "fde051860bb81ae434a5dcddae8812af"
server_key_version: "encryptionkey@0"
I20260812 02:46:27.955703 2984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:27.982882 2984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:27.987797 2984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:27.988117 2984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:27.989296 2984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:27.990784 2984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:27.990911 2984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:27.991047 2984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:27.991134 2984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:28.027581 2984 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:46121
I20260812 02:46:28.027683 3134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:46121 every 8 connection(s)
I20260812 02:46:28.029184 2984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:46:28.034617 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:28.032559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:53135)
0812 02:46:28.033306 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:28.033608 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":278,"threads_started":1}
W20260812 02:46:28.035327 3136 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:37967 (0 consecutive failures): Network error: Failed to ping master at 127.17.196.254:37967: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
I20260812 02:46:28.036230 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:28.035873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:35129)
0812 02:46:28.036024 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:28.036117 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 02:46:28.036908 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 2984
I20260812 02:46:28.037283 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:28.036851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:48017)
0812 02:46:28.037041 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:28.037141 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 02:46:28.037580 3136 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 02:46:28.037761 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:42517
--local_ip_for_outbound_sockets=127.17.196.195
--tserver_master_addrs=127.17.196.254:37967
--webserver_port=38391
--webserver_interface=127.17.196.195
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:28.385205 3141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:28.385546 3141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:28.385699 3141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:28.393180 3141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:28.393301 3141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:28.393509 3141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:46:28.403784 3141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:42517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=38391
--tserver_master_addrs=127.17.196.254:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:28.405193 3141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:28.406652 3141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:28.420668 3147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:28.420676 3149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:28.420698 3146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:28.421903 3141 server_base.cc:1034] running on GCE node
I20260812 02:46:28.422608 3141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:28.423713 3141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:28.424974 3141 hybrid_clock.cc:648] HybridClock initialized: now 1786502788424909 us; error 52 us; skew 500 ppm
I20260812 02:46:28.425369 3141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:28 dist-test-slave-ss4n krb5kdc[1427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502788, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:28.429086 3141 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:46:28.430963 3141 webserver.cc:460] Webserver started at http://127.17.196.195:38391/ using document root <none> and password file <none>
I20260812 02:46:28.431854 3141 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:28.432024 3141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:28.438501 3141 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:46:28.441684 3157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:28.443218 3141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:46:28.443424 3141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "6df02ad79c3847af92272dfc3c64bad2"
format_stamp: "Formatted at 2026-08-12 02:46:21 on dist-test-slave-ss4n"
server_key: "da3320dd20b745ae73a95dafdf5b71e6"
server_key_iv: "211cd0c66d163a8742a2594bdda28ee5"
server_key_version: "encryptionkey@0"
I20260812 02:46:28.444519 3141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:28.465114 3141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:28.470019 3141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:28.470322 3141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:28.471441 3141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:28.472934 3141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:28.473083 3141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:28.473217 3141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:28.473302 3141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:28.509531 3141 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:42517
I20260812 02:46:28.509589 3298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:42517 every 8 connection(s)
I20260812 02:46:28.511026 3141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:46:28.516418 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:28.515442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:37903)
0812 02:46:28.515949 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:28.516084 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":251,"threads_started":1}
I20260812 02:46:28.517174 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:28.514515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:41705)
0812 02:46:28.515635 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:28.515956 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":605,"threads_started":1}
W20260812 02:46:28.517974 3299 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:37967 (0 consecutive failures): Network error: Failed to ping master at 127.17.196.254:37967: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
I20260812 02:46:28.519019 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:28.518598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:59459)
0812 02:46:28.518799 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:28.518882 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 02:46:28.519896 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 3141
I20260812 02:46:28.520256 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:28.519765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:35477)
0812 02:46:28.519955 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:28.520080 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 02:46:28.520547 3299 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 02:46:29.038913 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:29.038064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:40345)
0812 02:46:29.038554 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:29.038677 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":219,"threads_started":1}
I20260812 02:46:29.518342 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:29.517469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:34685)
0812 02:46:29.518075 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:29.518169 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":301,"threads_started":1}
I20260812 02:46:29.522550 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:29.521330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:56185)
0812 02:46:29.522109 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:29.522276 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":364,"threads_started":1}
I20260812 02:46:30.041196 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:30.040190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:43823)
0812 02:46:30.040875 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:30.040997 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":355,"threads_started":1}
I20260812 02:46:30.520457 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:30.519455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:45419)
0812 02:46:30.520025 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:30.520190 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":279,"threads_started":1}
I20260812 02:46:30.524497 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:30.523486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:35751)
0812 02:46:30.524222 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:30.524313 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":325,"threads_started":1}
I20260812 02:46:31.043467 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:31.042398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:40055)
0812 02:46:31.043082 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:31.043226 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":389,"threads_started":1}
I20260812 02:46:31.522464 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:31.521672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:59335)
0812 02:46:31.522206 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:31.522296 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":281,"threads_started":1}
I20260812 02:46:31.526335 3314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:31.525394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:57739)
0812 02:46:31.526056 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:31.526152 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":429,"threads_started":1}
I20260812 02:46:32.045893 3315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:32.044847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:36029)
0812 02:46:32.045369 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:32.045467 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":258,"threads_started":1}
I20260812 02:46:32.524405 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:32.523529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:46951)
0812 02:46:32.524142 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:32.524228 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":312,"threads_started":1}
I20260812 02:46:32.528316 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:32.527284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:51251)
0812 02:46:32.527962 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:32.528066 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":411,"threads_started":1}
I20260812 02:46:33.047904 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:33.047027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:33833)
0812 02:46:33.047610 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:33.047708 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":272,"threads_started":1}
I20260812 02:46:33.526255 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:33.525389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:36875)
0812 02:46:33.525948 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:33.526064 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":321,"threads_started":1}
I20260812 02:46:33.530330 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:33.529330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:39867)
0812 02:46:33.530063 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:33.530151 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":462,"threads_started":1}
I20260812 02:46:34.049952 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:34.048995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:45515)
0812 02:46:34.049676 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:34.049781 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":332,"threads_started":1}
I20260812 02:46:34.528283 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:34.527315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:54873)
0812 02:46:34.527942 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:34.528047 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":297,"threads_started":1}
I20260812 02:46:34.532413 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:34.531460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:48091)
0812 02:46:34.532118 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:34.532233 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":321,"threads_started":1}
I20260812 02:46:35.052002 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:35.051033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:52953)
0812 02:46:35.051659 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:35.051760 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":244,"threads_started":1}
I20260812 02:46:35.530336 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:35.529413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:55719)
0812 02:46:35.530010 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:35.530145 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":354,"threads_started":1}
I20260812 02:46:35.534633 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:35.533583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:41855)
0812 02:46:35.534219 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:35.534312 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":253,"threads_started":1}
I20260812 02:46:36.054265 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:36.053243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:46439)
0812 02:46:36.053873 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:36.053999 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":338,"threads_started":1}
I20260812 02:46:36.532434 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:36.531480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:48327)
0812 02:46:36.532119 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:36.532228 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":349,"threads_started":1}
I20260812 02:46:36.536474 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:36.535738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:36455)
0812 02:46:36.536213 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:36.536298 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":245,"threads_started":1}
I20260812 02:46:37.056507 3334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:37.055649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:59827)
0812 02:46:37.056215 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:37.056312 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":283,"threads_started":1}
I20260812 02:46:37.534672 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:37.533773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:54009)
0812 02:46:37.534359 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:37.534474 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":281,"threads_started":1}
I20260812 02:46:37.538331 3336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:37.537556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:34683)
0812 02:46:37.538069 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:37.538153 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":262,"threads_started":1}
I20260812 02:46:38.058660 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:38.057716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:41225)
0812 02:46:38.058286 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:38.058390 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":298,"threads_started":1}
I20260812 02:46:38.536688 3338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:38.535831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:36805)
0812 02:46:38.536401 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:38.536502 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":332,"threads_started":1}
I20260812 02:46:38.540256 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:38.539364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:52691)
0812 02:46:38.539953 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:38.540047 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":349,"threads_started":1}
I20260812 02:46:39.060674 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:39.059740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:58061)
0812 02:46:39.060371 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:39.060485 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":383,"threads_started":1}
I20260812 02:46:39.538842 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:39.537901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:50265)
0812 02:46:39.538542 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:39.538654 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":327,"threads_started":1}
I20260812 02:46:39.542212 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:39.541338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:37001)
0812 02:46:39.541880 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:39.541992 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":313,"threads_started":1}
I20260812 02:46:40.062934 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:40.061996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:42081)
0812 02:46:40.062598 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:40.062717 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":318,"threads_started":1}
I20260812 02:46:40.541239 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:40.540235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:41611)
0812 02:46:40.540886 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:40.541019 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":354,"threads_started":1}
I20260812 02:46:40.544019 3347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:40.543243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:48399)
0812 02:46:40.543764 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:40.543851 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":276,"threads_started":1}
I20260812 02:46:41.065474 3350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:41.064162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:38859)
0812 02:46:41.065023 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:41.065178 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":405,"threads_started":1}
I20260812 02:46:41.543507 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:41.542424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:59181)
0812 02:46:41.543099 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:41.543232 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":351,"threads_started":1}
I20260812 02:46:41.545987 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:41.545004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:35375)
0812 02:46:41.545665 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:41.545772 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":286,"threads_started":1}
I20260812 02:46:42.067636 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:42.066738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:55503)
0812 02:46:42.067302 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:42.067413 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":262,"threads_started":1}
I20260812 02:46:42.545821 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:42.544875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:49499)
0812 02:46:42.545364 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:42.545466 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":242,"threads_started":1}
I20260812 02:46:42.547847 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:42.547030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:45305)
0812 02:46:42.547557 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:42.547657 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":234,"threads_started":1}
I20260812 02:46:43.069994 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:43.069085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.194:45619)
0812 02:46:43.069674 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:43.069804 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":248,"threads_started":1}
I20260812 02:46:43.523175 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 2789
I20260812 02:46:43.547966 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:43.546853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.193:58433)
0812 02:46:43.547532 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:43.547704 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":379,"threads_started":1}
I20260812 02:46:43.550146 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:43.549151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37967 (local address 127.17.196.195:51963)
0812 02:46:43.549707 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:43.549854 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:37967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":249,"threads_started":1}
I20260812 02:46:43.553040 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:43.524466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.17.196.193:43881)
0812 02:46:43.525172 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:43.525192 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:43.525243 (+ 51us) server_negotiation.cc:407] Connection header received
0812 02:46:43.527107 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:43.527118 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:43.527565 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:43.527870 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:43.529432 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:43.530871 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:43.532290 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:43.533080 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:43.533225 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:43.544497 (+ 11272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:43.544509 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:43.544529 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:43.544582 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:43.548898 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:43.550060 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:43.550068 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:43.550076 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:43.550337 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:43.550818 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:43.550822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:43.550824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:43.551457 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:43.551694 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:43.552237 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 02:46:43.552707 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":314,"threads_started":1}
I20260812 02:46:43.554695 2973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:43.644651 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 2789
I20260812 02:46:43.668596 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 2984
I20260812 02:46:43.689429 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:43.669822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59500 (local address 127.17.196.194:46121)
0812 02:46:43.670535 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:43.670554 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:43.670597 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:46:43.672369 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:43.672380 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:43.672856 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:43.673200 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:43.674714 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:43.676292 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:43.677660 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:43.678516 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:43.678695 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:43.681834 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:43.681847 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:43.681867 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:43.681923 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:43.685424 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:43.686452 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:43.686460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:43.686469 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:43.686756 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:43.687185 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:43.687190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:43.687192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:43.687943 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:43.688248 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:43.688698 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 02:46:43.689096 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":363,"threads_started":1}
I20260812 02:46:43.691107 3130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:43.777702 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 2984
I20260812 02:46:43.800992 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 3141
I20260812 02:46:43.821312 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:43.802387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.17.196.195:42517)
0812 02:46:43.803127 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:43.803146 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:43.803199 (+ 53us) server_negotiation.cc:407] Connection header received
0812 02:46:43.805054 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:43.805066 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:43.805782 (+ 716us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:43.806135 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:43.807484 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:43.808911 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:43.810406 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:43.811204 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:43.811364 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:43.814331 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:43.814340 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:43.814358 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:43.814404 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:43.817991 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:43.818692 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:43.818699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:43.818707 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:43.818903 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:43.819276 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:43.819279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:43.819282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:43.819878 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:43.820182 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:43.820468 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 02:46:43.820966 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":354,"threads_started":1}
I20260812 02:46:43.822875 3294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:43.911381 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 3141
2026-08-12T02:46:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25784 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:46:44 dist-test-slave-ss4n krb5kdc[3370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:46:44 dist-test-slave-ss4n krb5kdc[3370](info): set up 2 sockets
Aug 12 02:46:44 dist-test-slave-ss4n krb5kdc[3370](info): commencing operation
krb5kdc: starting...
W20260812 02:46:45.169641 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:46:45 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:46:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:46:45.503460 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:40393
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:40393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:45.886724 3385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:45.887042 3385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:45.887135 3385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:45.894798 3385 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 02:46:45.894907 3385 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 02:46:45.894946 3385 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 02:46:45.894991 3385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:46:45.895049 3385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:45.895120 3385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:45.895166 3385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:46:45.895216 3385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:46:45.906801 3385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:40393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:45.908214 3385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:45.909682 3385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:46:45.924479 3385 server_base.cc:1034] running on GCE node
W20260812 02:46:45.924392 3391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:45.924888 3393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:45.925125 3390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:45.926409 3385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:45.928176 3385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:45.929447 3385 hybrid_clock.cc:648] HybridClock initialized: now 1786502805929324 us; error 99 us; skew 500 ppm
I20260812 02:46:45.929896 3385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:45 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502805, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:45.933466 3385 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:46:45.935235 3385 webserver.cc:460] Webserver started at http://127.17.196.254:40883/ using document root <none> and password file <none>
I20260812 02:46:45.935956 3385 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:45.936053 3385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:45.936326 3385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:45.939574 3385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "011b8d833f7f45549f581fbb53633c3e"
format_stamp: "Formatted at 2026-08-12 02:46:45 on dist-test-slave-ss4n"
server_key: "ba90aa3b4450587e338e012095ce4d3e"
server_key_iv: "675f6caa676ca08a07031f90a35b1e20"
server_key_version: "encryptionkey@0"
I20260812 02:46:45.940377 3385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "011b8d833f7f45549f581fbb53633c3e"
format_stamp: "Formatted at 2026-08-12 02:46:45 on dist-test-slave-ss4n"
server_key: "ba90aa3b4450587e338e012095ce4d3e"
server_key_iv: "675f6caa676ca08a07031f90a35b1e20"
server_key_version: "encryptionkey@0"
I20260812 02:46:45.946589 3385 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 02:46:45.951119 3402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:45.952762 3385 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:46:45.953019 3385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "011b8d833f7f45549f581fbb53633c3e"
format_stamp: "Formatted at 2026-08-12 02:46:45 on dist-test-slave-ss4n"
server_key: "ba90aa3b4450587e338e012095ce4d3e"
server_key_iv: "675f6caa676ca08a07031f90a35b1e20"
server_key_version: "encryptionkey@0"
I20260812 02:46:45.953315 3385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:45.991442 3385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:45.996687 3385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:45.997015 3385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:46.018890 3385 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:40393
I20260812 02:46:46.018913 3459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:40393 every 8 connection(s)
I20260812 02:46:46.020406 3385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:46:46.024573 3460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:46.027477 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 3385
I20260812 02:46:46.027776 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:46:46.028586 18195 external_mini_cluster.cc:1428] Setting key 90ba80116e7a725419a42b0abfe46714
I20260812 02:46:46.038761 3460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Bootstrap starting.
I20260812 02:46:46.043216 3460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:46.044793 3460 log.cc:824] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:46:46 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:46.049172 3460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: No bootstrap required, opened a new log
I20260812 02:46:46.057798 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:46.031066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.17.196.254:40393)
0812 02:46:46.031993 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:46.032007 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:46.032052 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:46:46.034088 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:46.034125 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:46.034619 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:46.035012 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:46.037225 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:46.039088 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:46.041098 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:46.041969 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:46.042135 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:46.048170 (+ 6035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:46.048185 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:46.048208 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:46.048266 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:46.052376 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:46.053368 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:46.053376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:46.053385 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:46.053706 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:46.054190 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:46.054198 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:46.054211 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:46.054890 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:46.055286 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:46.055878 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 02:46:46.056418 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":432,"threads_started":1}
I20260812 02:46:46.058143 3460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:46:46.058686 3460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:46.058812 3460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011b8d833f7f45549f581fbb53633c3e, State: Initialized, Role: FOLLOWER
I20260812 02:46:46.059510 3460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:46:46.059815 3460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:46:46.059988 3460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:46:46.060199 3460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:46.063031 3460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:46:46.063613 3460 leader_election.cc:302] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 011b8d833f7f45549f581fbb53633c3e; no voters:
I20260812 02:46:46.064338 3460 leader_election.cc:288] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:46:46.064519 3465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:46.065377 3465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 1 LEADER]: Becoming Leader. State: Replica: 011b8d833f7f45549f581fbb53633c3e, State: Running, Role: LEADER
I20260812 02:46:46.066123 3465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:46:46.066943 3460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:46:46.069854 3466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "011b8d833f7f45549f581fbb53633c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } } }
I20260812 02:46:46.070287 3466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: This master's current role is: LEADER
I20260812 02:46:46.069976 3467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 011b8d833f7f45549f581fbb53633c3e. Latest consensus state: current_term: 1 leader_uuid: "011b8d833f7f45549f581fbb53633c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } } }
I20260812 02:46:46.070441 3467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: This master's current role is: LEADER
W20260812 02:46:46.089550 3484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:46:46.089798 3484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:46:46.089964 3473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:46:46.091944 3473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:46:46.101919 3473 catalog_manager.cc:1347] Generated new cluster ID: da81232013674738907146a315490525
I20260812 02:46:46.102079 3473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:46:46.116989 3473 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:46:46.118566 3473 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:46:46.133881 3473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Generated new TSK 0
I20260812 02:46:46.134950 3473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:46:46.226089 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:40393
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:46.618705 3489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:46.619093 3489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:46.619292 3489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:46.627689 3489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:46.627868 3489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:46.628082 3489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:46:46.639474 3489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:46.641036 3489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:46.642571 3489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:46.657316 3495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:46.657418 3499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:46.657291 3497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:46.658656 3489 server_base.cc:1034] running on GCE node
I20260812 02:46:46.659380 3489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:46.660490 3489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:46.661689 3489 hybrid_clock.cc:648] HybridClock initialized: now 1786502806661613 us; error 68 us; skew 500 ppm
I20260812 02:46:46.662048 3489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:46 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502806, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:46.665593 3489 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:46:46.667294 3489 webserver.cc:460] Webserver started at http://127.17.196.193:33617/ using document root <none> and password file <none>
I20260812 02:46:46.668133 3489 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:46.668286 3489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:46.668651 3489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:46.671805 3489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "4a32819549574f43a0300e85d0423795"
format_stamp: "Formatted at 2026-08-12 02:46:46 on dist-test-slave-ss4n"
server_key: "454971e41c7007c2b3f97608a320219e"
server_key_iv: "8ac03c6292230408d92d24edd2597846"
server_key_version: "encryptionkey@0"
I20260812 02:46:46.672636 3489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "4a32819549574f43a0300e85d0423795"
format_stamp: "Formatted at 2026-08-12 02:46:46 on dist-test-slave-ss4n"
server_key: "454971e41c7007c2b3f97608a320219e"
server_key_iv: "8ac03c6292230408d92d24edd2597846"
server_key_version: "encryptionkey@0"
I20260812 02:46:46.678256 3489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:46:46.681975 3507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:46.683368 3489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:46:46.683552 3489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "4a32819549574f43a0300e85d0423795"
format_stamp: "Formatted at 2026-08-12 02:46:46 on dist-test-slave-ss4n"
server_key: "454971e41c7007c2b3f97608a320219e"
server_key_iv: "8ac03c6292230408d92d24edd2597846"
server_key_version: "encryptionkey@0"
I20260812 02:46:46.683774 3489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:46.699138 3489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:46.703529 3489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:46.703861 3489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:46.704890 3489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:46.706496 3489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:46.706606 3489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:46.706758 3489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:46.706837 3489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:46.743142 3489 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:41523
I20260812 02:46:46.743180 3649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:41523 every 8 connection(s)
I20260812 02:46:46.744530 3489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:46:46.750502 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 3489
I20260812 02:46:46.750751 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:46:46.751643 18195 external_mini_cluster.cc:1428] Setting key 6f635bce365a2de899d35c22890a0bb4
Aug 12 02:46:46 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502806, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:46.771063 3652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:46.747208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:33765 (local address 127.17.196.254:40393)
0812 02:46:46.747739 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:46.747746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:46.748717 (+ 971us) server_negotiation.cc:407] Connection header received
0812 02:46:46.751048 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:46.751052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:46.751150 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:46.751294 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:46.753344 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:46.754292 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:46.757015 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:46.757704 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:46.757853 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:46.763244 (+ 5391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:46.763254 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:46.763261 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:46.763314 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:46.767120 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:46.768098 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:46.768106 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:46.768111 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:46.768334 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:46.768866 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:46.768874 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:46.768880 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:46.769287 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:46.769436 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:46.770284 (+ 848us) server_negotiation.cc:299] Negotiation successful
0812 02:46:46.770560 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":235,"threads_started":1}
I20260812 02:46:46.771765 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:46.747810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:33765)
0812 02:46:46.748541 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:46.748581 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:46.750667 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:46.751602 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:46.751614 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:46.752396 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:46.753158 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:46.753179 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:46.754486 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:46.754492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:46.756796 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:46.756816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:46.757979 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:46.757990 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:46.758371 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:46.759284 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:46.759322 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:46.762923 (+ 3601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:46.767295 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:46.767304 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:46.767319 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:46.767933 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:46.768490 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:46.768494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:46.768504 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:46.768729 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:46.769429 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:46.769435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:46.769881 (+ 446us) client_negotiation.cc:769] Sending connection context
0812 02:46:46.770143 (+ 262us) client_negotiation.cc:240] Negotiation successful
0812 02:46:46.770568 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":287,"threads_started":1}
I20260812 02:46:46.773538 3650 heartbeater.cc:347] Connected to a master server at 127.17.196.254:40393
I20260812 02:46:46.773880 3650 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:46.774726 3650 heartbeater.cc:511] Master 127.17.196.254:40393 requested a full tablet report, sending...
I20260812 02:46:46.776966 3419 ts_manager.cc:194] Registered new tserver with Master: 4a32819549574f43a0300e85d0423795 (127.17.196.193:41523)
I20260812 02:46:46.778555 3419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:33765
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:46:46.825333 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:40393
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:47.181057 3657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:47.181533 3657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:47.181728 3657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:47.189360 3657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:47.189471 3657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:47.189670 3657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:46:47.199816 3657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:47.201217 3657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:47.202638 3657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:46:47.219069 3664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:47.219053 3668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:47.219125 3665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:47.220006 3657 server_base.cc:1034] running on GCE node
I20260812 02:46:47.220716 3657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:47.221908 3657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:47.223130 3657 hybrid_clock.cc:648] HybridClock initialized: now 1786502807223063 us; error 42 us; skew 500 ppm
I20260812 02:46:47.223486 3657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:47 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502807, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:47.227061 3657 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:46:47.228780 3657 webserver.cc:460] Webserver started at http://127.17.196.194:32875/ using document root <none> and password file <none>
I20260812 02:46:47.229642 3657 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:47.229804 3657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:47.230145 3657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:47.233229 3657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "91e7f512c4f071f416c380b0d1cc46a2"
server_key_iv: "64eb395e699fbbc8483f447ea7bf65ad"
server_key_version: "encryptionkey@0"
I20260812 02:46:47.234129 3657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "91e7f512c4f071f416c380b0d1cc46a2"
server_key_iv: "64eb395e699fbbc8483f447ea7bf65ad"
server_key_version: "encryptionkey@0"
I20260812 02:46:47.239638 3657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:46:47.243594 3677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:47.245136 3657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:46:47.245334 3657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "91e7f512c4f071f416c380b0d1cc46a2"
server_key_iv: "64eb395e699fbbc8483f447ea7bf65ad"
server_key_version: "encryptionkey@0"
I20260812 02:46:47.245625 3657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:47.269176 3657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:47.274021 3657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:47.274323 3657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:47.275444 3657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:47.276930 3657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:47.277040 3657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:47.277199 3657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:47.277271 3657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:47.311720 3657 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:35539
I20260812 02:46:47.311820 3810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:35539 every 8 connection(s)
I20260812 02:46:47.313087 3657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:46:47.317363 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 3657
I20260812 02:46:47.317655 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:46:47.318149 18195 external_mini_cluster.cc:1428] Setting key bbcddf38eeda5bde3ce9aa9afbe66c88
Aug 12 02:46:47 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502807, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:47.338407 3813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.316135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:52223 (local address 127.17.196.254:40393)
0812 02:46:47.316626 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:47.316631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:47.317475 (+ 844us) server_negotiation.cc:407] Connection header received
0812 02:46:47.319750 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:47.319756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:47.319914 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:47.320029 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:47.322659 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.323509 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.326130 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.326568 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.326687 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.331292 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:47.331297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:47.331300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:47.331339 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:47.334582 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:47.335406 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:47.335414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:47.335421 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:47.335579 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:47.336176 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:47.336184 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:47.336190 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:47.336606 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:47.336764 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:47.337607 (+ 843us) server_negotiation.cc:299] Negotiation successful
0812 02:46:47.337886 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":261,"threads_started":1}
I20260812 02:46:47.339154 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.316558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:52223)
0812 02:46:47.317293 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:47.317340 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:47.319455 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:47.320496 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:47.320508 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:47.321421 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:47.322372 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.322407 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.323697 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.323705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.325940 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.325961 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.326829 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.326838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.327208 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.328057 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:47.328094 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:47.330978 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:47.334753 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:47.334770 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:47.334785 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:47.335253 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:47.335737 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:47.335742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:47.335744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:47.336012 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:47.336780 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:47.336787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:47.337155 (+ 368us) client_negotiation.cc:769] Sending connection context
0812 02:46:47.337407 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 02:46:47.337951 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":305,"threads_started":1}
I20260812 02:46:47.340802 3811 heartbeater.cc:347] Connected to a master server at 127.17.196.254:40393
I20260812 02:46:47.341205 3811 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:47.342211 3811 heartbeater.cc:511] Master 127.17.196.254:40393 requested a full tablet report, sending...
I20260812 02:46:47.344002 3419 ts_manager.cc:194] Registered new tserver with Master: 9945b3ce8bcb4c1ca9d4ee34c318cf8c (127.17.196.194:35539)
I20260812 02:46:47.344894 3419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:52223
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:46:47.393903 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:40393
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:46:47.754279 3817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:46:47.754628 3817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:46:47.754796 3817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:46:47.762567 3817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:46:47.762746 3817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:46:47.762925 3817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:46:47.773269 3817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:46:47.774808 3817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:46:47.776229 3817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:46:47.781888 3650 heartbeater.cc:503] Master 127.17.196.254:40393 was elected leader, sending a full tablet report...
W20260812 02:46:47.791137 3823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:47.791177 3825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:46:47.791011 3822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:46:47.792659 3817 server_base.cc:1034] running on GCE node
I20260812 02:46:47.793402 3817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:46:47.794534 3817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:46:47.795796 3817 hybrid_clock.cc:648] HybridClock initialized: now 1786502807795737 us; error 66 us; skew 500 ppm
I20260812 02:46:47.796178 3817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:46:47 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502807, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:46:47.799688 3817 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:46:47.801257 3817 webserver.cc:460] Webserver started at http://127.17.196.195:39767/ using document root <none> and password file <none>
I20260812 02:46:47.802095 3817 fs_manager.cc:359] Metadata directory not provided
I20260812 02:46:47.802278 3817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:46:47.802623 3817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:46:47.806463 3817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "23970e9a1a364fe89aad47d5ea5e32de"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "e982d7e570e746332b5ed8a405e532a2"
server_key_iv: "760aeac5f3f2bb8d7ae13c95a479b2ea"
server_key_version: "encryptionkey@0"
I20260812 02:46:47.807467 3817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "23970e9a1a364fe89aad47d5ea5e32de"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "e982d7e570e746332b5ed8a405e532a2"
server_key_iv: "760aeac5f3f2bb8d7ae13c95a479b2ea"
server_key_version: "encryptionkey@0"
I20260812 02:46:47.813694 3817 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 02:46:47.817669 3833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:47.819235 3817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:46:47.819450 3817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "23970e9a1a364fe89aad47d5ea5e32de"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "e982d7e570e746332b5ed8a405e532a2"
server_key_iv: "760aeac5f3f2bb8d7ae13c95a479b2ea"
server_key_version: "encryptionkey@0"
I20260812 02:46:47.819732 3817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:46:47.846022 3817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:46:47.850924 3817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:46:47.851284 3817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:46:47.852478 3817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:46:47.854012 3817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:46:47.854094 3817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:47.854231 3817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:46:47.854283 3817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:46:47.889951 3817 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:46711
I20260812 02:46:47.890003 3966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:46711 every 8 connection(s)
I20260812 02:46:47.891352 3817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:46:47.897748 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 3817
I20260812 02:46:47.897998 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:46:47.898578 18195 external_mini_cluster.cc:1428] Setting key c3a8fdcf5acd6c190174f28e2fcf1888
Aug 12 02:46:47 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502807, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:46:47.913297 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.894230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:52317 (local address 127.17.196.254:40393)
0812 02:46:47.894709 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:47.894714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:47.895723 (+ 1009us) server_negotiation.cc:407] Connection header received
0812 02:46:47.897716 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:47.897724 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:47.897829 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:47.897967 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:47.900135 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.901072 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.902971 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.903375 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.903476 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.907138 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:47.907143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:47.907147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:47.907183 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:47.909902 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:47.910777 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:47.910781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:47.910783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:47.910917 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:47.911339 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:47.911342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:47.911344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:47.911625 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:47.911744 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:47.912668 (+ 924us) server_negotiation.cc:299] Negotiation successful
0812 02:46:47.912911 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":207,"threads_started":1}
I20260812 02:46:47.914700 3970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.894810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:52317)
0812 02:46:47.895544 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:47.895592 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:47.897379 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:47.898287 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:47.898299 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:47.899085 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:46:47.899917 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.899946 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.901241 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.901246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.902830 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.902844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.903585 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.903590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.903837 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.904517 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:46:47.904541 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:46:47.906906 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:46:47.910065 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:47.910074 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:47.910089 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:47.910609 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:47.911017 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:46:47.911021 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:46:47.911023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:46:47.911228 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:46:47.911738 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:46:47.911744 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:46:47.912139 (+ 395us) client_negotiation.cc:769] Sending connection context
0812 02:46:47.912486 (+ 347us) client_negotiation.cc:240] Negotiation successful
0812 02:46:47.913081 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":298,"threads_started":1}
I20260812 02:46:47.916600 3967 heartbeater.cc:347] Connected to a master server at 127.17.196.254:40393
I20260812 02:46:47.917007 3967 heartbeater.cc:464] Registering TS with master...
I20260812 02:46:47.918058 3967 heartbeater.cc:511] Master 127.17.196.254:40393 requested a full tablet report, sending...
I20260812 02:46:47.919997 3419 ts_manager.cc:194] Registered new tserver with Master: 23970e9a1a364fe89aad47d5ea5e32de (127.17.196.195:46711)
I20260812 02:46:47.921213 3419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:52317
I20260812 02:46:47.927826 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:46:47.946242 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.932586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.17.196.254:40393)
0812 02:46:47.932772 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:47.932777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:47.933224 (+ 447us) server_negotiation.cc:407] Connection header received
0812 02:46:47.933366 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:47.933369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:47.933451 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:47.933810 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:47.935242 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.935981 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.937376 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.937837 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.937935 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.940455 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:47.940460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:47.940463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:47.940492 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:47.943311 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:47.944010 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:47.944016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:47.944020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:47.944177 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:47.944575 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:47.944581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:47.944584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:47.944966 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:47.945227 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:47.945646 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 02:46:47.945889 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:46:47.950667 3419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:46:47.957882 3419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:46:47.998994 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.973804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.17.196.193:41523)
0812 02:46:47.979337 (+ 5533us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:47.979352 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:47.989532 (+ 10180us) server_negotiation.cc:407] Connection header received
0812 02:46:47.989842 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:47.989850 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:47.990091 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:47.990276 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:47.991228 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.992824 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.996114 (+ 3290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.997554 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.997736 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.997896 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:47.998484 (+ 588us) server_negotiation.cc:299] Negotiation successful
0812 02:46:47.998669 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5270,"thread_start_us":5073,"threads_started":1}
I20260812 02:46:47.999040 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.972843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:41523 (local address 127.0.0.1:39940)
0812 02:46:47.989387 (+ 16544us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:47.989404 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:47.989641 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:47.990588 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:47.990592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:47.990617 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:47.991016 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.991030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.992985 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.992991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.995919 (+ 2928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.995935 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.998087 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.998093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.998280 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.998317 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:46:47.998433 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 02:46:47.998566 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16433,"mutex_wait_us":207,"thread_start_us":15115,"threads_started":1}
I20260812 02:46:48.000367 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.970950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35539 (local address 127.0.0.1:56852)
0812 02:46:47.987268 (+ 16318us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:47.987311 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:47.987530 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:47.991783 (+ 4253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:47.991788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:47.991816 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:47.992295 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.992307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.994259 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.994265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.996699 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.996714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.999542 (+ 2828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.999554 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.999755 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.999788 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:46:47.999862 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 02:46:48.000026 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16047,"thread_start_us":15887,"threads_started":1}
I20260812 02:46:48.000728 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.971168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:46711 (local address 127.0.0.1:44592)
0812 02:46:47.988297 (+ 17129us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:47.988320 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:47.988461 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:47.989837 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:47.989844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:47.989875 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:47.990373 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.990388 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.992985 (+ 2597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.992991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.995648 (+ 2657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:47.995662 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.997791 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.997799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:47.998045 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.998109 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 02:46:47.998268 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 02:46:47.998494 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16953,"mutex_wait_us":82,"thread_start_us":17964,"threads_started":1}
I20260812 02:46:48.001407 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.971679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.17.196.195:46711)
0812 02:46:47.989081 (+ 17402us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:47.989091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:47.989133 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:46:47.989246 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:47.989254 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:47.989576 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:47.990588 (+ 1012us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:47.991067 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.992832 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.995881 (+ 3049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.997147 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.997372 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:47.997637 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:47.999039 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0812 02:46:48.001032 (+ 1993us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17208,"thread_start_us":17054,"threads_started":1}
I20260812 02:46:48.001581 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:47.971207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.17.196.194:35539)
0812 02:46:47.991272 (+ 20065us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:47.991279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:47.991304 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:46:47.991382 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:47.991387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:47.991556 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:47.991797 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:47.992514 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.994064 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:47.997207 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:47.998838 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.000722 (+ 1884us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.000921 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:48.001008 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 02:46:48.001198 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19892,"thread_start_us":19773,"threads_started":1}
I20260812 02:46:48.004072 3573 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1380c26ce149bbbb499610171bf027 (DEFAULT_TABLE table=test-table [id=11e1a9b34c7a4b54a8895a008860d441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:48.005127 3735 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1380c26ce149bbbb499610171bf027 (DEFAULT_TABLE table=test-table [id=11e1a9b34c7a4b54a8895a008860d441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:48.005780 3573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1380c26ce149bbbb499610171bf027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:48.005395 3893 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1380c26ce149bbbb499610171bf027 (DEFAULT_TABLE table=test-table [id=11e1a9b34c7a4b54a8895a008860d441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:46:48.006605 3735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1380c26ce149bbbb499610171bf027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:48.006861 3893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1380c26ce149bbbb499610171bf027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:46:48.022837 3988 tablet_bootstrap.cc:492] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: Bootstrap starting.
I20260812 02:46:48.029002 3986 tablet_bootstrap.cc:492] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Bootstrap starting.
I20260812 02:46:48.032328 3988 tablet_bootstrap.cc:654] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:48.033308 3987 tablet_bootstrap.cc:492] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Bootstrap starting.
I20260812 02:46:48.033535 3986 tablet_bootstrap.cc:654] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:48.034793 3986 log.cc:824] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:48.037246 3987 tablet_bootstrap.cc:654] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Neither blocks nor log segments found. Creating new log.
I20260812 02:46:48.037842 3986 tablet_bootstrap.cc:492] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: No bootstrap required, opened a new log
I20260812 02:46:48.038161 3986 ts_tablet_manager.cc:1397] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260812 02:46:48.039011 3988 log.cc:824] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:48.040100 3987 log.cc:824] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Log is configured to *not* fsync() on all Append() calls
I20260812 02:46:48.049853 3988 tablet_bootstrap.cc:492] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: No bootstrap required, opened a new log
I20260812 02:46:48.050263 3988 ts_tablet_manager.cc:1397] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.003s
I20260812 02:46:48.051151 3986 raft_consensus.cc:348] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.051512 3987 tablet_bootstrap.cc:492] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: No bootstrap required, opened a new log
I20260812 02:46:48.051646 3986 raft_consensus.cc:374] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:48.051759 3986 raft_consensus.cc:729] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9945b3ce8bcb4c1ca9d4ee34c318cf8c, State: Initialized, Role: FOLLOWER
I20260812 02:46:48.051877 3987 ts_tablet_manager.cc:1397] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.010s
I20260812 02:46:48.052506 3986 consensus_queue.cc:260] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.054328 3811 heartbeater.cc:503] Master 127.17.196.254:40393 was elected leader, sending a full tablet report...
I20260812 02:46:48.055047 3986 ts_tablet_manager.cc:1428] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
I20260812 02:46:48.062299 3988 raft_consensus.cc:348] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.062361 3987 raft_consensus.cc:348] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.062786 3988 raft_consensus.cc:374] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:48.062901 3988 raft_consensus.cc:729] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a32819549574f43a0300e85d0423795, State: Initialized, Role: FOLLOWER
I20260812 02:46:48.062901 3987 raft_consensus.cc:374] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:46:48.063040 3987 raft_consensus.cc:729] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23970e9a1a364fe89aad47d5ea5e32de, State: Initialized, Role: FOLLOWER
I20260812 02:46:48.063589 3988 consensus_queue.cc:260] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.063789 3987 consensus_queue.cc:260] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.065065 3967 heartbeater.cc:503] Master 127.17.196.254:40393 was elected leader, sending a full tablet report...
W20260812 02:46:48.065199 3812 tablet.cc:2369] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:48.065591 3988 ts_tablet_manager.cc:1428] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260812 02:46:48.067186 3987 ts_tablet_manager.cc:1428] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
W20260812 02:46:48.143136 3968 tablet.cc:2369] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:46:48.148212 3994 raft_consensus.cc:482] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:46:48.148479 3994 raft_consensus.cc:504] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.150398 3994 leader_election.cc:288] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9945b3ce8bcb4c1ca9d4ee34c318cf8c (127.17.196.194:35539), 23970e9a1a364fe89aad47d5ea5e32de (127.17.196.195:46711)
I20260812 02:46:48.160243 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:48.150828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:48641 (local address 127.17.196.194:35539)
0812 02:46:48.151067 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:48.151074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:48.152084 (+ 1010us) server_negotiation.cc:407] Connection header received
0812 02:46:48.152300 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:48.152305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:48.152401 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:48.152497 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:48.153116 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.154291 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.157459 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.158929 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.159114 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.159216 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:48.159785 (+ 569us) server_negotiation.cc:299] Negotiation successful
0812 02:46:48.159940 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:46:48.160244 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:48.151016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:35555 (local address 127.17.196.195:46711)
0812 02:46:48.151231 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:48.151239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:48.151994 (+ 755us) server_negotiation.cc:407] Connection header received
0812 02:46:48.152272 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:48.152294 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:48.152386 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:48.152491 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:46:48.153266 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.154347 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.157459 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.158930 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.159149 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.159223 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:48.159785 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 02:46:48.159941 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 02:46:48.160418 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:48.150833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35539 (local address 127.17.196.193:48641)
0812 02:46:48.151976 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:48.151995 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:48.152117 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:48.152512 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:48.152516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:48.152540 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:48.152925 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:48.152934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.154468 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.154474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:48.157209 (+ 2735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:48.157226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.159244 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.159250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:48.159521 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.159630 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 02:46:48.159729 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 02:46:48.159906 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1021,"mutex_wait_us":322,"thread_start_us":491,"threads_started":1}
I20260812 02:46:48.160914 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:48.150814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:46711 (local address 127.17.196.193:35555)
0812 02:46:48.151863 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0812 02:46:48.151880 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:46:48.152086 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:46:48.152647 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:46:48.152650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:46:48.152682 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:46:48.153075 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:48.153127 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.154494 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.154501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:48.157209 (+ 2708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:46:48.157226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.159223 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.159231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:46:48.159518 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.159627 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 02:46:48.159729 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 02:46:48.159906 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":161,"thread_start_us":569,"threads_started":1}
I20260812 02:46:48.161685 3758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1380c26ce149bbbb499610171bf027" candidate_uuid: "4a32819549574f43a0300e85d0423795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" is_pre_election: true
I20260812 02:46:48.161844 3916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1380c26ce149bbbb499610171bf027" candidate_uuid: "4a32819549574f43a0300e85d0423795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23970e9a1a364fe89aad47d5ea5e32de" is_pre_election: true
I20260812 02:46:48.162331 3758 raft_consensus.cc:2454] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a32819549574f43a0300e85d0423795 in term 0.
I20260812 02:46:48.162372 3916 raft_consensus.cc:2454] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a32819549574f43a0300e85d0423795 in term 0.
I20260812 02:46:48.163074 3509 leader_election.cc:302] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23970e9a1a364fe89aad47d5ea5e32de, 4a32819549574f43a0300e85d0423795; no voters:
I20260812 02:46:48.163486 3994 raft_consensus.cc:2781] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:46:48.163640 3994 raft_consensus.cc:482] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:46:48.163753 3994 raft_consensus.cc:3037] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:48.166038 3994 raft_consensus.cc:504] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.166712 3994 leader_election.cc:288] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [CANDIDATE]: Term 1 election: Requested vote from peers 9945b3ce8bcb4c1ca9d4ee34c318cf8c (127.17.196.194:35539), 23970e9a1a364fe89aad47d5ea5e32de (127.17.196.195:46711)
I20260812 02:46:48.167480 3758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1380c26ce149bbbb499610171bf027" candidate_uuid: "4a32819549574f43a0300e85d0423795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c"
I20260812 02:46:48.167380 3916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1380c26ce149bbbb499610171bf027" candidate_uuid: "4a32819549574f43a0300e85d0423795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23970e9a1a364fe89aad47d5ea5e32de"
I20260812 02:46:48.167724 3758 raft_consensus.cc:3037] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:48.167728 3916 raft_consensus.cc:3037] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:46:48.171482 3758 raft_consensus.cc:2454] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a32819549574f43a0300e85d0423795 in term 1.
I20260812 02:46:48.171482 3916 raft_consensus.cc:2454] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a32819549574f43a0300e85d0423795 in term 1.
I20260812 02:46:48.172040 3509 leader_election.cc:302] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 23970e9a1a364fe89aad47d5ea5e32de, 4a32819549574f43a0300e85d0423795, 9945b3ce8bcb4c1ca9d4ee34c318cf8c; no voters:
I20260812 02:46:48.172498 3994 raft_consensus.cc:2781] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:46:48.173014 3994 raft_consensus.cc:686] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 1 LEADER]: Becoming Leader. State: Replica: 4a32819549574f43a0300e85d0423795, State: Running, Role: LEADER
I20260812 02:46:48.173745 3994 consensus_queue.cc:237] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:46:48.178733 3419 catalog_manager.cc:5495] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a32819549574f43a0300e85d0423795 (127.17.196.193). New cstate: current_term: 1 leader_uuid: "4a32819549574f43a0300e85d0423795" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } health_report { overall_health: HEALTHY } } }
I20260812 02:46:48.207343 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:48.200254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39956 (local address 127.17.196.193:41523)
0812 02:46:48.200540 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:48.200545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:48.200672 (+ 127us) server_negotiation.cc:407] Connection header received
0812 02:46:48.200880 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:48.200884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:48.200990 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:48.201147 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:46:48.201874 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.202709 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.204647 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.205139 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.205257 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.206034 (+ 777us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:46:48.206643 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:46:48.206772 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:48.206914 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 02:46:48.206995 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 02:46:48.217832 3916 raft_consensus.cc:1261] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 1 FOLLOWER]: Refusing update from remote peer 4a32819549574f43a0300e85d0423795: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:48.217911 3758 raft_consensus.cc:1261] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 1 FOLLOWER]: Refusing update from remote peer 4a32819549574f43a0300e85d0423795: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:46:48.219029 3998 consensus_queue.cc:1035] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:48.219466 3994 consensus_queue.cc:1035] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:46:48.229856 4003 mvcc.cc:205] Tried to move back new op lower bound from 7317515502448959488 to 7317515502283890688. Current Snapshot: MvccSnapshot[applied={T|T < 7317515502448959488}]
I20260812 02:46:48.232096 4004 mvcc.cc:205] Tried to move back new op lower bound from 7317515502448959488 to 7317515502283890688. Current Snapshot: MvccSnapshot[applied={T|T < 7317515502448959488}]
I20260812 02:46:48.249202 3419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:46:48.249620 3419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:46:48.254284 3419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Sending DeleteTablet for 3 replicas of tablet 9d1380c26ce149bbbb499610171bf027
I20260812 02:46:48.255716 3893 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1380c26ce149bbbb499610171bf027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:48 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:44592
I20260812 02:46:48.255810 3573 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1380c26ce149bbbb499610171bf027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:48 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:39940
I20260812 02:46:48.255807 3735 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1380c26ce149bbbb499610171bf027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:46:48 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:56852
I20260812 02:46:48.257236 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 3489
I20260812 02:46:48.266876 4013 tablet_replica.cc:330] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: stopping tablet replica
I20260812 02:46:48.267622 4013 raft_consensus.cc:2229] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:46:48.268414 4013 raft_consensus.cc:2258] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:48.271293 4014 tablet_replica.cc:330] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: stopping tablet replica
I20260812 02:46:48.272007 4014 raft_consensus.cc:2229] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:46:48.273048 4014 raft_consensus.cc:2258] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:48.274703 4015 tablet_replica.cc:330] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: stopping tablet replica
I20260812 02:46:48.276288 4013 ts_tablet_manager.cc:1905] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:48.276458 4014 ts_tablet_manager.cc:1905] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:48.281826 4015 raft_consensus.cc:2229] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:46:48.282502 4015 raft_consensus.cc:2258] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:46:48.284235 4013 ts_tablet_manager.cc:1918] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:48.284456 4013 log.cc:1196] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/9d1380c26ce149bbbb499610171bf027
I20260812 02:46:48.285120 4013 ts_tablet_manager.cc:1939] T 9d1380c26ce149bbbb499610171bf027 P 23970e9a1a364fe89aad47d5ea5e32de: Deleting consensus metadata
I20260812 02:46:48.286190 4015 ts_tablet_manager.cc:1905] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:46:48.287461 3404 catalog_manager.cc:4841] TS 23970e9a1a364fe89aad47d5ea5e32de (127.17.196.195:46711): tablet 9d1380c26ce149bbbb499610171bf027 (table test-table [id=11e1a9b34c7a4b54a8895a008860d441]) successfully deleted
Aug 12 02:46:48 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:46:48.291831 4014 ts_tablet_manager.cc:1918] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:48.292052 4014 log.cc:1196] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/9d1380c26ce149bbbb499610171bf027
I20260812 02:46:48.292737 4014 ts_tablet_manager.cc:1939] T 9d1380c26ce149bbbb499610171bf027 P 4a32819549574f43a0300e85d0423795: Deleting consensus metadata
I20260812 02:46:48.295118 3403 catalog_manager.cc:4841] TS 4a32819549574f43a0300e85d0423795 (127.17.196.193:41523): tablet 9d1380c26ce149bbbb499610171bf027 (table test-table [id=11e1a9b34c7a4b54a8895a008860d441]) successfully deleted
I20260812 02:46:48.295336 4015 ts_tablet_manager.cc:1918] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:46:48.295547 4015 log.cc:1196] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/9d1380c26ce149bbbb499610171bf027
I20260812 02:46:48.296181 4015 ts_tablet_manager.cc:1939] T 9d1380c26ce149bbbb499610171bf027 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Deleting consensus metadata
I20260812 02:46:48.298421 3405 catalog_manager.cc:4841] TS 9945b3ce8bcb4c1ca9d4ee34c318cf8c (127.17.196.194:35539): tablet 9d1380c26ce149bbbb499610171bf027 (table test-table [id=11e1a9b34c7a4b54a8895a008860d441]) successfully deleted
I20260812 02:46:48.300005 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:48.258806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39970 (local address 127.17.196.193:41523)
0812 02:46:48.259011 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:48.259023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:48.272876 (+ 13853us) server_negotiation.cc:407] Connection header received
0812 02:46:48.273588 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:48.273601 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:48.273722 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:48.273833 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:48.276916 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.278321 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.280814 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.283159 (+ 2345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.283326 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.291271 (+ 7945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:48.291293 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:48.291314 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:48.291403 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:48.296355 (+ 4952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:48.297287 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:48.297308 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:48.297310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:48.297470 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:48.298003 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:48.298010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:48.298013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:48.298596 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:48.298924 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:48.299309 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 02:46:48.299587 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1464,"server-negotiator.queue_time_us":72}
I20260812 02:46:48.300834 3642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:48.409046 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 3489
I20260812 02:46:48.431362 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 3657
Aug 12 02:46:48 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:46:48.448246 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:48.432388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56866 (local address 127.17.196.194:35539)
0812 02:46:48.432611 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:48.432618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:48.432634 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:46:48.432690 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:48.432693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:48.432766 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:48.432883 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:48.434452 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.435222 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.436548 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.436942 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.437086 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.441674 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:48.441691 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:48.441708 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:48.441753 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:48.444887 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:48.445868 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:48.445877 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:48.445879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:48.446032 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:48.446498 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:48.446502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:48.446504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:48.446956 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:48.447273 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:48.447625 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 02:46:48.447873 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:46:48.449075 3804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:48.544070 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 3657
I20260812 02:46:48.566684 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 3817
Aug 12 02:46:48 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:46:48.588586 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:46:48.567684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.17.196.195:46711)
0812 02:46:48.567914 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:46:48.567919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:46:48.567939 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:46:48.568169 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:46:48.568174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:46:48.568274 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:46:48.568370 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:46:48.569867 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.570574 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.572049 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:46:48.572619 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:46:48.572752 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:46:48.581896 (+ 9144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:46:48.581915 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:46:48.581932 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:46:48.581984 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:46:48.585193 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:48.586161 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:48.586173 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:48.586179 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:48.586374 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:46:48.586865 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:46:48.586870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:46:48.586872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:46:48.587336 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:46:48.587612 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 02:46:48.587985 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 02:46:48.588171 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 02:46:48.589452 3960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:48.680989 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 3817
I20260812 02:46:48.702163 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 3385
I20260812 02:46:48.703490 3454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:46:48.816430 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 3385
I20260812 02:47:08.846280 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:40393
--webserver_interface=127.17.196.254
--webserver_port=40883
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:40393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:47:09.219151 4021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:09.219465 4021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:09.219556 4021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:09.227145 4021 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 02:47:09.227270 4021 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 02:47:09.227312 4021 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 02:47:09.227357 4021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:47:09.227415 4021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:09.227487 4021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:09.227533 4021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:47:09.227589 4021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:47:09.238179 4021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:40393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=40883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:09.239516 4021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:09.240912 4021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:09.269445 4028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:09.269336 4031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:09.269284 4029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:09.270331 4021 server_base.cc:1034] running on GCE node
I20260812 02:47:09.271410 4021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:09.273258 4021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:09.274533 4021 hybrid_clock.cc:648] HybridClock initialized: now 1786502829274444 us; error 113 us; skew 500 ppm
I20260812 02:47:09.275022 4021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:09 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502829, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:09.279157 4021 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:47:09.281163 4021 webserver.cc:460] Webserver started at http://127.17.196.254:40883/ using document root <none> and password file <none>
I20260812 02:47:09.282130 4021 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:09.282318 4021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:09.289397 4021 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:47:09.293970 4039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:09.296010 4021 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:47:09.296285 4021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "011b8d833f7f45549f581fbb53633c3e"
format_stamp: "Formatted at 2026-08-12 02:46:45 on dist-test-slave-ss4n"
server_key: "ba90aa3b4450587e338e012095ce4d3e"
server_key_iv: "675f6caa676ca08a07031f90a35b1e20"
server_key_version: "encryptionkey@0"
I20260812 02:47:09.297610 4021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:09.320441 4021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:09.327072 4021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:09.327452 4021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:09.351046 4021 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:40393
I20260812 02:47:09.351079 4111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:40393 every 8 connection(s)
I20260812 02:47:09.352911 4021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:47:09.353688 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 4021
I20260812 02:47:09.354928 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:41523
--local_ip_for_outbound_sockets=127.17.196.193
--tserver_master_addrs=127.17.196.254:40393
--webserver_port=33617
--webserver_interface=127.17.196.193
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:47:09.362987 4112 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:47:09.368525 4112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Bootstrap starting.
I20260812 02:47:09.390846 4112 log.cc:824] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:09.405120 4112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:47:09.406033 4112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Bootstrap complete.
I20260812 02:47:09.418185 4112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:47:09.419274 4112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011b8d833f7f45549f581fbb53633c3e, State: Initialized, Role: FOLLOWER
I20260812 02:47:09.420255 4112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:47:09.420596 4112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:47:09.420778 4112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:47:09.420996 4112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:47:09.425585 4112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:47:09.426213 4112 leader_election.cc:302] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 011b8d833f7f45549f581fbb53633c3e; no voters:
I20260812 02:47:09.426936 4112 leader_election.cc:288] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 02:47:09.427109 4116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:47:09.428416 4116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 2 LEADER]: Becoming Leader. State: Replica: 011b8d833f7f45549f581fbb53633c3e, State: Running, Role: LEADER
I20260812 02:47:09.429044 4116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:47:09.429709 4112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:47:09.432041 4119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 011b8d833f7f45549f581fbb53633c3e. Latest consensus state: current_term: 2 leader_uuid: "011b8d833f7f45549f581fbb53633c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } } }
I20260812 02:47:09.432471 4119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: This master's current role is: LEADER
I20260812 02:47:09.433106 4117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "011b8d833f7f45549f581fbb53633c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } } }
I20260812 02:47:09.433456 4117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: This master's current role is: LEADER
I20260812 02:47:09.442284 4124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:47:09.453437 4124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:47:09.455621 4124 catalog_manager.cc:1259] Loaded cluster ID: da81232013674738907146a315490525
I20260812 02:47:09.455776 4124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:47:09.461746 4124 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:47:09.464097 4124 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Loaded TSK: 0
I20260812 02:47:09.527617 4124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Generated new TSK 1
I20260812 02:47:09.528178 4124 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:47:09.813047 4114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:09.813459 4114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:09.813789 4114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:09.822654 4114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:09.822849 4114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:09.823046 4114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:47:09.835392 4114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:41523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=33617
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:09.836917 4114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:09.838433 4114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:09.854839 4148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:09.854794 4149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:09.855644 4151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:09.856078 4114 server_base.cc:1034] running on GCE node
I20260812 02:47:09.856813 4114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:09.858093 4114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:09.859365 4114 hybrid_clock.cc:648] HybridClock initialized: now 1786502829859296 us; error 61 us; skew 500 ppm
I20260812 02:47:09.859778 4114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:09 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502829, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:09.863906 4114 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:47:09.865896 4114 webserver.cc:460] Webserver started at http://127.17.196.193:33617/ using document root <none> and password file <none>
I20260812 02:47:09.866710 4114 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:09.866883 4114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:09.873610 4114 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.006s
I20260812 02:47:09.877032 4159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:09.878768 4114 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:47:09.879041 4114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "4a32819549574f43a0300e85d0423795"
format_stamp: "Formatted at 2026-08-12 02:46:46 on dist-test-slave-ss4n"
server_key: "454971e41c7007c2b3f97608a320219e"
server_key_iv: "8ac03c6292230408d92d24edd2597846"
server_key_version: "encryptionkey@0"
I20260812 02:47:09.880179 4114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:09.905072 4114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:09.910768 4114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:09.911093 4114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:09.912321 4114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:09.914143 4114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:09.914297 4114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:09.914451 4114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:09.914577 4114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 02:47:09.957683 4114 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:41523
I20260812 02:47:09.957731 4297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:41523 every 8 connection(s)
I20260812 02:47:09.959172 4114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:47:09.963073 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 4114
I20260812 02:47:09.963969 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:35539
--local_ip_for_outbound_sockets=127.17.196.194
--tserver_master_addrs=127.17.196.254:40393
--webserver_port=32875
--webserver_interface=127.17.196.194
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:47:09 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502829, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:47:09.985378 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:09.963034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:39755)
0812 02:47:09.963898 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:09.963951 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:09.965991 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:09.967454 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:09.967469 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:09.968141 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:09.968697 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:09.968715 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:09.970565 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:09.970570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:09.971827 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:09.971841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:09.972952 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:09.972960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:09.973240 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:09.974003 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:47:09.974033 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:09.976540 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:09.980408 (+ 3868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:09.980418 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:09.980434 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:09.980901 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:09.981407 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:09.981412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:09.981416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:09.981713 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:09.982577 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:09.982585 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:09.982964 (+ 379us) client_negotiation.cc:769] Sending connection context
0812 02:47:09.983258 (+ 294us) client_negotiation.cc:240] Negotiation successful
0812 02:47:09.983882 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":341,"threads_started":1}
I20260812 02:47:09.985380 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:09.962643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:39755 (local address 127.17.196.254:40393)
0812 02:47:09.963273 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:09.963286 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:09.964098 (+ 812us) server_negotiation.cc:407] Connection header received
0812 02:47:09.966564 (+ 2466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:09.966577 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:09.966918 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:09.967288 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:09.968916 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:09.970345 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:09.972005 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:09.972673 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:09.972892 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:09.976870 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:09.976883 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:09.976908 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:09.976974 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:09.980233 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:09.981067 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:09.981076 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:09.981085 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:09.981276 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:09.981837 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:09.981843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:09.981847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:09.982415 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:09.982643 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:09.983507 (+ 864us) server_negotiation.cc:299] Negotiation successful
0812 02:47:09.983918 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":314,"threads_started":1}
I20260812 02:47:09.988602 4298 heartbeater.cc:347] Connected to a master server at 127.17.196.254:40393
I20260812 02:47:09.988972 4298 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:09.989836 4298 heartbeater.cc:511] Master 127.17.196.254:40393 requested a full tablet report, sending...
I20260812 02:47:09.992175 4064 ts_manager.cc:194] Registered new tserver with Master: 4a32819549574f43a0300e85d0423795 (127.17.196.193:41523)
I20260812 02:47:09.993803 4064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:39755
W20260812 02:47:10.330864 4304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:10.331305 4304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:10.331476 4304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:10.339165 4304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:10.339356 4304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:10.339540 4304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:47:10.349828 4304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:35539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=32875
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:10.351325 4304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:10.352764 4304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:10.367733 4310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:10.367762 4309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:10.368180 4313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:10.368820 4304 server_base.cc:1034] running on GCE node
I20260812 02:47:10.369580 4304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:10.370671 4304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:10.371912 4304 hybrid_clock.cc:648] HybridClock initialized: now 1786502830371869 us; error 35 us; skew 500 ppm
I20260812 02:47:10.372253 4304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:10 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502830, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:10.375684 4304 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:47:10.377892 4304 webserver.cc:460] Webserver started at http://127.17.196.194:32875/ using document root <none> and password file <none>
I20260812 02:47:10.378695 4304 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:10.378870 4304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:10.384840 4304 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 02:47:10.387948 4321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:10.389907 4304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:47:10.390151 4304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "91e7f512c4f071f416c380b0d1cc46a2"
server_key_iv: "64eb395e699fbbc8483f447ea7bf65ad"
server_key_version: "encryptionkey@0"
I20260812 02:47:10.391360 4304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:10.424897 4304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:10.430709 4304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:10.431057 4304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:10.432372 4304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:10.434144 4304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:10.434294 4304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:10.434453 4304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:10.434556 4304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:10.476943 4304 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:35539
I20260812 02:47:10.477015 4460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:35539 every 8 connection(s)
I20260812 02:47:10.478850 4304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:47:10.481348 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 4304
I20260812 02:47:10.482170 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:46711
--local_ip_for_outbound_sockets=127.17.196.195
--tserver_master_addrs=127.17.196.254:40393
--webserver_port=39767
--webserver_interface=127.17.196.195
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:47:10 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502830, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:47:10.508951 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:10.482190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:57495 (local address 127.17.196.254:40393)
0812 02:47:10.482441 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:10.482448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:10.484056 (+ 1608us) server_negotiation.cc:407] Connection header received
0812 02:47:10.487235 (+ 3179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:10.487240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:10.487340 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:10.487447 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:10.490036 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:10.490932 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:10.493827 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:10.494400 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:10.494577 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:10.499520 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:10.499534 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:10.499543 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:10.499607 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:10.504152 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:10.505542 (+ 1390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:10.505552 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:10.505559 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:10.505753 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:10.506372 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:10.506382 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:10.506388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:10.506863 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:10.507095 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:10.508091 (+ 996us) server_negotiation.cc:299] Negotiation successful
0812 02:47:10.508418 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 02:47:10.509788 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:10.482847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:57495)
0812 02:47:10.483814 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:10.483877 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:10.486840 (+ 2963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:10.487778 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:10.487794 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:10.488696 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:10.489722 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:10.489757 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:10.491185 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:10.491193 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:10.493589 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:10.493612 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:10.494707 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:10.494718 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:10.495078 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:10.496085 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:47:10.496114 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:10.499118 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:10.504578 (+ 5460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:10.504590 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:10.504608 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:10.505312 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:10.505927 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:10.505931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:10.505934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:10.506203 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:10.507192 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:10.507199 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:10.507597 (+ 398us) client_negotiation.cc:769] Sending connection context
0812 02:47:10.507889 (+ 292us) client_negotiation.cc:240] Negotiation successful
0812 02:47:10.508377 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":365,"threads_started":1}
I20260812 02:47:10.511682 4461 heartbeater.cc:347] Connected to a master server at 127.17.196.254:40393
I20260812 02:47:10.512077 4461 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:10.512928 4461 heartbeater.cc:511] Master 127.17.196.254:40393 requested a full tablet report, sending...
I20260812 02:47:10.514715 4064 ts_manager.cc:194] Registered new tserver with Master: 9945b3ce8bcb4c1ca9d4ee34c318cf8c (127.17.196.194:35539)
I20260812 02:47:10.515610 4064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:57495
W20260812 02:47:10.858808 4465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:10.859198 4465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:10.859364 4465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:10.868582 4465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:10.868744 4465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:10.868877 4465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:47:10.880771 4465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:46711
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=39767
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:10.882308 4465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:10.883888 4465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:10.899526 4471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:10.899680 4473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:10.899641 4470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:10.901151 4465 server_base.cc:1034] running on GCE node
I20260812 02:47:10.902113 4465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:10.903342 4465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:10.904774 4465 hybrid_clock.cc:648] HybridClock initialized: now 1786502830904682 us; error 76 us; skew 500 ppm
I20260812 02:47:10.905333 4465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:10 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502830, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:10.910146 4465 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:47:10.912334 4465 webserver.cc:460] Webserver started at http://127.17.196.195:39767/ using document root <none> and password file <none>
I20260812 02:47:10.913167 4465 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:10.913349 4465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:10.920341 4465 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:47:10.924260 4485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:10.926254 4465 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:47:10.926517 4465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "23970e9a1a364fe89aad47d5ea5e32de"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "e982d7e570e746332b5ed8a405e532a2"
server_key_iv: "760aeac5f3f2bb8d7ae13c95a479b2ea"
server_key_version: "encryptionkey@0"
I20260812 02:47:10.927757 4465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:10.951283 4465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:10.957231 4465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:10.957580 4465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:10.958788 4465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:10.960354 4465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:10.960460 4465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:10.960635 4465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:10.960723 4465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:10.997125 4298 heartbeater.cc:503] Master 127.17.196.254:40393 was elected leader, sending a full tablet report...
I20260812 02:47:11.002130 4465 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:46711
I20260812 02:47:11.002224 4614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:46711 every 8 connection(s)
I20260812 02:47:11.003965 4465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:47:11.013710 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 4465
Aug 12 02:47:11 dist-test-slave-ss4n krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502830, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:47:11.028967 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.007317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:35279 (local address 127.17.196.254:40393)
0812 02:47:11.007661 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.007667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.008998 (+ 1331us) server_negotiation.cc:407] Connection header received
0812 02:47:11.012179 (+ 3181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.012185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.012304 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.012426 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:11.014847 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.016084 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.017890 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.018359 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.018510 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.022552 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:11.022557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:11.022562 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:11.022596 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:11.025579 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.026398 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.026404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.026408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.026572 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.027057 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.027072 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.027074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.027354 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:11.027513 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.028260 (+ 747us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.028527 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 02:47:11.030558 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.007887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:35279)
0812 02:47:11.008783 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:11.008829 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:11.011761 (+ 2932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:11.012752 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:11.012763 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:11.013687 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:11.014625 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.014651 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.016301 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.016306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.017725 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.017740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.018647 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.018656 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.019023 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.019792 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:47:11.019823 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:11.022308 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:11.025738 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:11.025748 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:11.025769 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:11.026256 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:11.026701 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:11.026706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:11.026710 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:11.026926 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:11.027489 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:11.027498 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:11.027887 (+ 389us) client_negotiation.cc:769] Sending connection context
0812 02:47:11.028173 (+ 286us) client_negotiation.cc:240] Negotiation successful
0812 02:47:11.028754 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":358,"threads_started":1}
I20260812 02:47:11.032668 4615 heartbeater.cc:347] Connected to a master server at 127.17.196.254:40393
I20260812 02:47:11.033044 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.015004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.17.196.254:40393)
0812 02:47:11.015640 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.015647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.016012 (+ 365us) server_negotiation.cc:407] Connection header received
0812 02:47:11.016096 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.016101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.016183 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.016364 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:11.018663 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.019603 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.021344 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.022025 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.022212 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.025583 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:11.025590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:11.025594 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:11.025630 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:11.028870 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.030105 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.030113 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.030121 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.030301 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.031144 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.031148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.031150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.031586 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:11.031895 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.032303 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.032572 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":296,"threads_started":1}
I20260812 02:47:11.033191 4615 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:11.034583 4615 heartbeater.cc:511] Master 127.17.196.254:40393 requested a full tablet report, sending...
I20260812 02:47:11.036499 4064 ts_manager.cc:194] Registered new tserver with Master: 23970e9a1a364fe89aad47d5ea5e32de (127.17.196.195:46711)
I20260812 02:47:11.037532 4064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:35279
I20260812 02:47:11.050019 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:47:11.058620 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.051356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.17.196.254:40393)
0812 02:47:11.051684 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.051691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.052060 (+ 369us) server_negotiation.cc:407] Connection header received
0812 02:47:11.052364 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.052368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.052545 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.052758 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:47:11.053355 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.054210 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.056053 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.056582 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.056743 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.057366 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:47:11.057792 (+ 426us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 02:47:11.058258 (+ 466us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33210: token expired
Metrics: {"server-negotiator.queue_time_us":143}
W20260812 02:47:11.058818 4620 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33210: token expired
W20260812 02:47:11.059536 3972 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.196.254:40393)
I20260812 02:47:11.075923 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.060279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.17.196.254:40393)
0812 02:47:11.060612 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.060618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.060642 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:47:11.060687 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.060690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.060759 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.060874 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:11.062684 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.063511 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.065185 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.065778 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.065935 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.069120 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:11.069126 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:11.069129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:11.069174 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:11.072518 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.073533 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.073539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.073545 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.073734 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.074179 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.074186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.074189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.074563 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:11.074844 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.075243 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.075514 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:47:11.080468 4064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:47:11.089148 4064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:47:11.117149 4630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.105094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40238 (local address 127.17.196.194:35539)
0812 02:47:11.105847 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.105867 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.106722 (+ 855us) server_negotiation.cc:407] Connection header received
0812 02:47:11.107267 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.107280 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.107584 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.107775 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:11.109210 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.110808 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.114165 (+ 3357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.115589 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.115827 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.116064 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.116558 (+ 494us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.116777 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":314,"threads_started":1}
I20260812 02:47:11.117282 4629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.104627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35539 (local address 127.0.0.1:40238)
0812 02:47:11.106518 (+ 1891us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:11.106580 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:11.106912 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:11.108195 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:11.108203 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:11.108264 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:11.108946 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.108962 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.110977 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.110983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.113925 (+ 2942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.113946 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.115868 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.115875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.116145 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.116246 (+ 101us) client_negotiation.cc:769] Sending connection context
0812 02:47:11.116451 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 02:47:11.116675 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1595,"mutex_wait_us":74,"spinlock_wait_cycles":9984,"thread_start_us":2193,"threads_started":1}
I20260812 02:47:11.118384 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.104649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:46711 (local address 127.0.0.1:55272)
0812 02:47:11.106777 (+ 2128us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:11.106798 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:11.107046 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:11.108190 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:11.108200 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:11.108248 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:11.109162 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.109178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.111551 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.111557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.114219 (+ 2662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.114231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.116267 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.116273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.117636 (+ 1363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.117683 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 02:47:11.117784 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 02:47:11.117939 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1993,"mutex_wait_us":602,"spinlock_wait_cycles":3968,"thread_start_us":1085,"threads_started":1}
I20260812 02:47:11.118783 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.104776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.17.196.195:46711)
0812 02:47:11.106272 (+ 1496us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.106281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.106905 (+ 624us) server_negotiation.cc:407] Connection header received
0812 02:47:11.107286 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.107296 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.107584 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.107825 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:11.109371 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.111372 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.114401 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.115912 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.116147 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.116336 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.118126 (+ 1790us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.118344 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1302,"thread_start_us":1054,"threads_started":1}
I20260812 02:47:11.119568 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.107045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:41523 (local address 127.0.0.1:40668)
0812 02:47:11.108030 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:11.108051 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:11.108190 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:11.109679 (+ 1489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:11.109685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:11.109714 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:11.110095 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.110108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.112172 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.112178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.114767 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.114784 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.118587 (+ 3803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.118600 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.118816 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.118863 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 02:47:11.118989 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 02:47:11.119163 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":7,"thread_start_us":943,"threads_started":1}
I20260812 02:47:11.119946 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.107774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.17.196.193:41523)
0812 02:47:11.108931 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.108941 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.108970 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:47:11.109093 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.109102 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.109385 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.109684 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:11.110288 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.111953 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.114971 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.116747 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.116971 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.117174 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.119054 (+ 1880us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.119265 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":876,"threads_started":1}
I20260812 02:47:11.122135 4376 tablet_service.cc:1467] Processing CreateTablet for tablet 6ff252a7dd0047f28f90d2b1c2357a95 (DEFAULT_TABLE table=test-table [id=4e839cfc2ad248f5b7580fb41aa75163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:11.123101 4538 tablet_service.cc:1467] Processing CreateTablet for tablet 6ff252a7dd0047f28f90d2b1c2357a95 (DEFAULT_TABLE table=test-table [id=4e839cfc2ad248f5b7580fb41aa75163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:11.123972 4376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ff252a7dd0047f28f90d2b1c2357a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:11.124570 4538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ff252a7dd0047f28f90d2b1c2357a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:11.124281 4224 tablet_service.cc:1467] Processing CreateTablet for tablet 6ff252a7dd0047f28f90d2b1c2357a95 (DEFAULT_TABLE table=test-table [id=4e839cfc2ad248f5b7580fb41aa75163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:11.125844 4224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ff252a7dd0047f28f90d2b1c2357a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:11.146134 4637 tablet_bootstrap.cc:492] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Bootstrap starting.
I20260812 02:47:11.147379 4635 tablet_bootstrap.cc:492] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Bootstrap starting.
I20260812 02:47:11.148440 4636 tablet_bootstrap.cc:492] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Bootstrap starting.
I20260812 02:47:11.150908 4637 tablet_bootstrap.cc:654] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:11.152710 4637 log.cc:824] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:11.153683 4636 tablet_bootstrap.cc:654] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:11.153793 4635 tablet_bootstrap.cc:654] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:11.155196 4635 log.cc:824] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:11.155527 4636 log.cc:824] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:11.159492 4635 tablet_bootstrap.cc:492] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: No bootstrap required, opened a new log
I20260812 02:47:11.159833 4635 ts_tablet_manager.cc:1397] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.006s
I20260812 02:47:11.164028 4637 tablet_bootstrap.cc:492] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: No bootstrap required, opened a new log
I20260812 02:47:11.164363 4637 ts_tablet_manager.cc:1397] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.007s
I20260812 02:47:11.168938 4635 raft_consensus.cc:348] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.169396 4635 raft_consensus.cc:374] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:11.169627 4635 raft_consensus.cc:729] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23970e9a1a364fe89aad47d5ea5e32de, State: Initialized, Role: FOLLOWER
I20260812 02:47:11.170467 4635 consensus_queue.cc:260] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.171801 4615 heartbeater.cc:503] Master 127.17.196.254:40393 was elected leader, sending a full tablet report...
I20260812 02:47:11.172569 4635 ts_tablet_manager.cc:1428] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260812 02:47:11.174207 4636 tablet_bootstrap.cc:492] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: No bootstrap required, opened a new log
I20260812 02:47:11.174597 4636 ts_tablet_manager.cc:1397] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.004s
I20260812 02:47:11.174613 4637 raft_consensus.cc:348] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.175172 4637 raft_consensus.cc:374] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:11.175292 4637 raft_consensus.cc:729] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9945b3ce8bcb4c1ca9d4ee34c318cf8c, State: Initialized, Role: FOLLOWER
I20260812 02:47:11.176124 4637 consensus_queue.cc:260] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.177862 4461 heartbeater.cc:503] Master 127.17.196.254:40393 was elected leader, sending a full tablet report...
I20260812 02:47:11.178300 4637 ts_tablet_manager.cc:1428] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260812 02:47:11.184597 4636 raft_consensus.cc:348] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.185154 4636 raft_consensus.cc:374] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:11.185308 4636 raft_consensus.cc:729] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a32819549574f43a0300e85d0423795, State: Initialized, Role: FOLLOWER
I20260812 02:47:11.186059 4636 consensus_queue.cc:260] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.187879 4636 ts_tablet_manager.cc:1428] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260812 02:47:11.211885 4299 tablet.cc:2369] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:47:11.231226 4463 tablet.cc:2369] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:47:11.256016 4617 tablet.cc:2369] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:47:11.375736 4644 raft_consensus.cc:482] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:47:11.376065 4644 raft_consensus.cc:504] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.378181 4644 leader_election.cc:288] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23970e9a1a364fe89aad47d5ea5e32de (127.17.196.195:46711), 9945b3ce8bcb4c1ca9d4ee34c318cf8c (127.17.196.194:35539)
I20260812 02:47:11.386620 4630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.378846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:34253 (local address 127.17.196.194:35539)
0812 02:47:11.379096 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.379105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.379961 (+ 856us) server_negotiation.cc:407] Connection header received
0812 02:47:11.380273 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.380279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.380371 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.380483 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:11.381270 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.382198 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.384413 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.385319 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.385552 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.385665 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.386106 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.386289 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:47:11.386616 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.378730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:43389 (local address 127.17.196.195:46711)
0812 02:47:11.379089 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.379100 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.379735 (+ 635us) server_negotiation.cc:407] Connection header received
0812 02:47:11.380023 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.380044 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.380150 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.380262 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:11.381094 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.382065 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.384309 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.385356 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.385615 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.385697 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.386166 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.386325 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 02:47:11.386821 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.378618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:35539 (local address 127.17.196.193:34253)
0812 02:47:11.379846 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:11.379866 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:11.380101 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:11.380512 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:11.380516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:11.380546 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:11.381075 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.381089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.382353 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.382357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.384142 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.384157 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.385616 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.385627 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.385916 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.385945 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:47:11.386036 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 02:47:11.386214 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":27,"thread_start_us":533,"threads_started":1}
I20260812 02:47:11.386893 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.378548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:46711 (local address 127.17.196.193:43389)
0812 02:47:11.379551 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:11.379589 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:11.379807 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:11.380295 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:11.380300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:11.380323 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:11.380888 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.380901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.382236 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.382245 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.384101 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:11.384119 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.385709 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.385719 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:11.385962 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.385990 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 02:47:11.386118 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 02:47:11.386289 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":31,"thread_start_us":924,"threads_started":1}
I20260812 02:47:11.388140 4563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ff252a7dd0047f28f90d2b1c2357a95" candidate_uuid: "4a32819549574f43a0300e85d0423795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23970e9a1a364fe89aad47d5ea5e32de" is_pre_election: true
I20260812 02:47:11.388140 4398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ff252a7dd0047f28f90d2b1c2357a95" candidate_uuid: "4a32819549574f43a0300e85d0423795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" is_pre_election: true
I20260812 02:47:11.388705 4398 raft_consensus.cc:2454] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a32819549574f43a0300e85d0423795 in term 0.
I20260812 02:47:11.388720 4563 raft_consensus.cc:2454] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a32819549574f43a0300e85d0423795 in term 0.
I20260812 02:47:11.389564 4163 leader_election.cc:302] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a32819549574f43a0300e85d0423795, 9945b3ce8bcb4c1ca9d4ee34c318cf8c; no voters:
I20260812 02:47:11.390035 4644 raft_consensus.cc:2781] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:47:11.390237 4644 raft_consensus.cc:482] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:47:11.390369 4644 raft_consensus.cc:3037] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:11.392832 4644 raft_consensus.cc:504] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.393606 4644 leader_election.cc:288] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [CANDIDATE]: Term 1 election: Requested vote from peers 23970e9a1a364fe89aad47d5ea5e32de (127.17.196.195:46711), 9945b3ce8bcb4c1ca9d4ee34c318cf8c (127.17.196.194:35539)
I20260812 02:47:11.394287 4398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ff252a7dd0047f28f90d2b1c2357a95" candidate_uuid: "4a32819549574f43a0300e85d0423795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c"
I20260812 02:47:11.394291 4563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ff252a7dd0047f28f90d2b1c2357a95" candidate_uuid: "4a32819549574f43a0300e85d0423795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23970e9a1a364fe89aad47d5ea5e32de"
I20260812 02:47:11.394526 4398 raft_consensus.cc:3037] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:11.394558 4563 raft_consensus.cc:3037] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:11.397223 4398 raft_consensus.cc:2454] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a32819549574f43a0300e85d0423795 in term 1.
I20260812 02:47:11.397419 4563 raft_consensus.cc:2454] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a32819549574f43a0300e85d0423795 in term 1.
I20260812 02:47:11.397981 4163 leader_election.cc:302] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a32819549574f43a0300e85d0423795, 9945b3ce8bcb4c1ca9d4ee34c318cf8c; no voters:
I20260812 02:47:11.398366 4644 raft_consensus.cc:2781] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:47:11.398875 4644 raft_consensus.cc:686] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 1 LEADER]: Becoming Leader. State: Replica: 4a32819549574f43a0300e85d0423795, State: Running, Role: LEADER
I20260812 02:47:11.399418 4644 consensus_queue.cc:237] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } }
I20260812 02:47:11.405292 4064 catalog_manager.cc:5495] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a32819549574f43a0300e85d0423795 (127.17.196.193). New cstate: current_term: 1 leader_uuid: "4a32819549574f43a0300e85d0423795" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a32819549574f43a0300e85d0423795" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 41523 } health_report { overall_health: HEALTHY } } }
I20260812 02:47:11.491856 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.485987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.17.196.193:41523)
0812 02:47:11.486220 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.486227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.486346 (+ 119us) server_negotiation.cc:407] Connection header received
0812 02:47:11.486616 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.486620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.486705 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.486868 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:47:11.487350 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.488102 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.489556 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.489962 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.490112 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.490644 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:47:11.491200 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:47:11.491376 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.491483 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.491588 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:47:11.501408 4563 raft_consensus.cc:1261] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 1 FOLLOWER]: Refusing update from remote peer 4a32819549574f43a0300e85d0423795: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:47:11.501452 4398 raft_consensus.cc:1261] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 1 FOLLOWER]: Refusing update from remote peer 4a32819549574f43a0300e85d0423795: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:47:11.502667 4644 consensus_queue.cc:1035] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23970e9a1a364fe89aad47d5ea5e32de" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 46711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:47:11.503041 4647 consensus_queue.cc:1035] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:47:11.533277 4063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:47:11.533774 4063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:47:11.538419 4063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Sending DeleteTablet for 3 replicas of tablet 6ff252a7dd0047f28f90d2b1c2357a95
I20260812 02:47:11.539841 4538 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ff252a7dd0047f28f90d2b1c2357a95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:55272
I20260812 02:47:11.540050 4224 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ff252a7dd0047f28f90d2b1c2357a95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:40668
I20260812 02:47:11.540094 4376 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ff252a7dd0047f28f90d2b1c2357a95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:40238
I20260812 02:47:11.540907 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 4114
I20260812 02:47:11.552451 4660 tablet_replica.cc:330] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: stopping tablet replica
I20260812 02:47:11.553562 4660 raft_consensus.cc:2229] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:47:11.554453 4660 raft_consensus.cc:2258] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:11.555495 4662 tablet_replica.cc:330] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: stopping tablet replica
I20260812 02:47:11.555964 4662 raft_consensus.cc:2229] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:47:11.556478 4662 raft_consensus.cc:2258] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:11.556764 4660 ts_tablet_manager.cc:1905] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:47:11.557596 4661 tablet_replica.cc:330] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: stopping tablet replica
I20260812 02:47:11.558097 4661 raft_consensus.cc:2229] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:47:11.558945 4662 ts_tablet_manager.cc:1905] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:47:11.559485 4661 raft_consensus.cc:2258] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:11.562270 4661 ts_tablet_manager.cc:1905] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:47:11.566661 4662 ts_tablet_manager.cc:1918] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:11.566951 4662 log.cc:1196] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/6ff252a7dd0047f28f90d2b1c2357a95
I20260812 02:47:11.567250 4660 ts_tablet_manager.cc:1918] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:11.567811 4662 ts_tablet_manager.cc:1939] T 6ff252a7dd0047f28f90d2b1c2357a95 P 9945b3ce8bcb4c1ca9d4ee34c318cf8c: Deleting consensus metadata
I20260812 02:47:11.570058 4043 catalog_manager.cc:4841] TS 9945b3ce8bcb4c1ca9d4ee34c318cf8c (127.17.196.194:35539): tablet 6ff252a7dd0047f28f90d2b1c2357a95 (table test-table [id=4e839cfc2ad248f5b7580fb41aa75163]) successfully deleted
I20260812 02:47:11.570935 4661 ts_tablet_manager.cc:1918] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:11.570961 4660 log.cc:1196] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/6ff252a7dd0047f28f90d2b1c2357a95
I20260812 02:47:11.571256 4661 log.cc:1196] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/6ff252a7dd0047f28f90d2b1c2357a95
I20260812 02:47:11.571646 4660 ts_tablet_manager.cc:1939] T 6ff252a7dd0047f28f90d2b1c2357a95 P 23970e9a1a364fe89aad47d5ea5e32de: Deleting consensus metadata
I20260812 02:47:11.572088 4661 ts_tablet_manager.cc:1939] T 6ff252a7dd0047f28f90d2b1c2357a95 P 4a32819549574f43a0300e85d0423795: Deleting consensus metadata
I20260812 02:47:11.574327 4040 catalog_manager.cc:4841] TS 4a32819549574f43a0300e85d0423795 (127.17.196.193:41523): tablet 6ff252a7dd0047f28f90d2b1c2357a95 (table test-table [id=4e839cfc2ad248f5b7580fb41aa75163]) successfully deleted
I20260812 02:47:11.574582 4042 catalog_manager.cc:4841] TS 23970e9a1a364fe89aad47d5ea5e32de (127.17.196.195:46711): tablet 6ff252a7dd0047f28f90d2b1c2357a95 (table test-table [id=4e839cfc2ad248f5b7580fb41aa75163]) successfully deleted
I20260812 02:47:11.579730 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.542245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.17.196.193:41523)
0812 02:47:11.542489 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.542501 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.550226 (+ 7725us) server_negotiation.cc:407] Connection header received
0812 02:47:11.550889 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.550902 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.551049 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.551158 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:11.554820 (+ 3662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.557666 (+ 2846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.560091 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.560924 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.561062 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.570991 (+ 9929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:11.571009 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:11.571029 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:11.571098 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:11.575504 (+ 4406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.576673 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.576682 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.576684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.576893 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.577646 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.577651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.577654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.578091 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:11.578347 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.578972 (+ 625us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.579226 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:47:11.580716 4292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:11.685885 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 4114
I20260812 02:47:11.711644 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 4304
I20260812 02:47:11.727495 4630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.712577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.17.196.194:35539)
0812 02:47:11.712815 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.712822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.712846 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:47:11.713061 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.713066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.713136 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.713292 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:11.714888 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.715652 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.717073 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.717624 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.717802 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.720791 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:11.720817 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:11.720834 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:11.720874 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:11.724213 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.725069 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.725077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.725079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.725216 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.725739 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.725743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.725745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.726152 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:11.726517 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.726852 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.727112 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 02:47:11.728259 4454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:11.823935 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 4304
I20260812 02:47:11.848555 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 4465
I20260812 02:47:11.864490 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:11.849471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.17.196.195:46711)
0812 02:47:11.849949 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:11.849954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:11.849973 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:47:11.850038 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:11.850042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:11.850107 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:11.850197 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:11.851749 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.852654 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.854291 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:11.854766 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:11.854874 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:11.857799 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:11.857816 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:11.857835 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:11.857876 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:11.860981 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.862088 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.862096 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.862099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.862264 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:11.862813 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:11.862817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:11.862819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:11.863302 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:11.863558 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:11.863899 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 02:47:11.864098 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:47:11.865279 4610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:11.965134 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 4465
I20260812 02:47:11.988492 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 4021
I20260812 02:47:11.989871 4105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:12.113814 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 4021
I20260812 02:47:32.143282 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:40393
--webserver_interface=127.17.196.254
--webserver_port=40883
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:40393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:47:32.528751 4670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:32.529152 4670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:32.529333 4670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:32.536831 4670 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 02:47:32.537003 4670 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 02:47:32.537103 4670 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 02:47:32.537204 4670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:47:32.537308 4670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:32.537417 4670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:32.537532 4670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:47:32.537621 4670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:47:32.548054 4670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:40393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=40883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:32.549525 4670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:32.551134 4670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:32.565004 4675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:32.564947 4676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:32.565027 4678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:32.565307 4670 server_base.cc:1034] running on GCE node
I20260812 02:47:32.566497 4670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:32.568063 4670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:32.569343 4670 hybrid_clock.cc:648] HybridClock initialized: now 1786502852569261 us; error 59 us; skew 500 ppm
I20260812 02:47:32.569742 4670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:32 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502852, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:32.577829 4670 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:47:32.579560 4670 webserver.cc:460] Webserver started at http://127.17.196.254:40883/ using document root <none> and password file <none>
I20260812 02:47:32.580372 4670 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:32.580520 4670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:32.586511 4670 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:47:32.589728 4685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:32.591140 4670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:47:32.591342 4670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "011b8d833f7f45549f581fbb53633c3e"
format_stamp: "Formatted at 2026-08-12 02:46:45 on dist-test-slave-ss4n"
server_key: "ba90aa3b4450587e338e012095ce4d3e"
server_key_iv: "675f6caa676ca08a07031f90a35b1e20"
server_key_version: "encryptionkey@0"
I20260812 02:47:32.592396 4670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:32.606776 4670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:32.611719 4670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:32.612087 4670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:32.634095 4670 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:40393
I20260812 02:47:32.634116 4752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:40393 every 8 connection(s)
I20260812 02:47:32.635890 4670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:47:32.640359 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 4670
I20260812 02:47:32.641327 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:41523
--local_ip_for_outbound_sockets=127.17.196.193
--tserver_master_addrs=127.17.196.254:40393
--webserver_port=33617
--webserver_interface=127.17.196.193
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:47:32.643697 4753 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:47:32.647606 4753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Bootstrap starting.
I20260812 02:47:32.669799 4753 log.cc:824] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:32.686812 4753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:47:32.687422 4753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Bootstrap complete.
I20260812 02:47:32.697333 4753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:47:32.698477 4753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011b8d833f7f45549f581fbb53633c3e, State: Initialized, Role: FOLLOWER
I20260812 02:47:32.699173 4753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:47:32.699429 4753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:47:32.699584 4753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:47:32.699774 4753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 2 FOLLOWER]: Advancing to term 3
I20260812 02:47:32.704250 4753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:47:32.704836 4753 leader_election.cc:302] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 011b8d833f7f45549f581fbb53633c3e; no voters:
I20260812 02:47:32.705657 4753 leader_election.cc:288] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 02:47:32.705816 4759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 02:47:32.706954 4759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [term 3 LEADER]: Becoming Leader. State: Replica: 011b8d833f7f45549f581fbb53633c3e, State: Running, Role: LEADER
I20260812 02:47:32.707652 4759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } }
I20260812 02:47:32.708036 4753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:47:32.710382 4761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 011b8d833f7f45549f581fbb53633c3e. Latest consensus state: current_term: 3 leader_uuid: "011b8d833f7f45549f581fbb53633c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } } }
I20260812 02:47:32.710479 4760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "011b8d833f7f45549f581fbb53633c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011b8d833f7f45549f581fbb53633c3e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 40393 } } }
I20260812 02:47:32.710906 4760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: This master's current role is: LEADER
I20260812 02:47:32.710791 4761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e [sys.catalog]: This master's current role is: LEADER
I20260812 02:47:32.712065 4767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:47:32.729164 4767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:47:32.731321 4767 catalog_manager.cc:1259] Loaded cluster ID: da81232013674738907146a315490525
I20260812 02:47:32.731541 4767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:47:32.736217 4767 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:47:32.739153 4767 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Loaded TSK: 0
I20260812 02:47:32.739427 4767 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 011b8d833f7f45549f581fbb53633c3e: Loaded TSK: 1
F20260812 02:47:32.740787 4767 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3b28ab39ad google::LogMessage::Fail() at ??:0
@ 0x7f3b28ab688a google::LogMessage::SendToLog() at ??:0
@ 0x7f3b28ab34de google::LogMessage::Flush() at ??:0
@ 0x7f3b28ab4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3b2c8177d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3b2c80673e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3b3e4e21c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3b3e4e1783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3b3e56d2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
I20260812 02:47:32.790046 4786 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f3b3e56d0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f3b3e5b75e9 std::function<>::operator()() at ??:0
@ 0x7f3b3e4e5cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3b3e4e3c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3b3e56b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3b3e56b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3b2b6ff345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3b2b70291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3b2b702799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3b2b6ddeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3b2e9b76db start_thread at ??:0
@ 0x7f3b2794d61f clone at ??:0
W20260812 02:47:33.082306 4756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:33.082638 4756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:33.082796 4756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:33.090866 4756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:33.091001 4756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:33.091189 4756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:47:33.101706 4756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:41523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=33617
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:33.103138 4756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:33.104555 4756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:33.119412 4812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:33.119458 4813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:33.120060 4815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:33.120355 4756 server_base.cc:1034] running on GCE node
I20260812 02:47:33.121048 4756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:33.122188 4756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:33.123481 4756 hybrid_clock.cc:648] HybridClock initialized: now 1786502853123409 us; error 53 us; skew 500 ppm
I20260812 02:47:33.123893 4756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:33 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502853, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:33.127663 4756 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:47:33.129326 4756 webserver.cc:460] Webserver started at http://127.17.196.193:33617/ using document root <none> and password file <none>
I20260812 02:47:33.130265 4756 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:33.130429 4756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:33.136647 4756 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 02:47:33.140024 4822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:33.141865 4756 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:47:33.142105 4756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "4a32819549574f43a0300e85d0423795"
format_stamp: "Formatted at 2026-08-12 02:46:46 on dist-test-slave-ss4n"
server_key: "454971e41c7007c2b3f97608a320219e"
server_key_iv: "8ac03c6292230408d92d24edd2597846"
server_key_version: "encryptionkey@0"
I20260812 02:47:33.143373 4756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:33.175017 4756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:33.179797 4756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:33.180155 4756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:33.181310 4756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:33.182885 4756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:33.183028 4756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:33.183171 4756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:33.183269 4756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:33.221562 4756 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:41523
I20260812 02:47:33.221661 4954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:41523 every 8 connection(s)
I20260812 02:47:33.223140 4756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:47:33.228541 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:33.226485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:54801)
0812 02:47:33.227343 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:33.227622 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":339,"threads_started":1}
W20260812 02:47:33.229415 4956 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:40393 (0 consecutive failures): Network error: Failed to ping master at 127.17.196.254:40393: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
I20260812 02:47:33.230777 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:33.230102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:36869)
0812 02:47:33.230357 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:33.230584 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 02:47:33.230855 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 4756
I20260812 02:47:33.231856 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:35539
--local_ip_for_outbound_sockets=127.17.196.194
--tserver_master_addrs=127.17.196.254:40393
--webserver_port=32875
--webserver_interface=127.17.196.194
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:47:33.232121 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:33.231554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:41065)
0812 02:47:33.231777 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:33.231906 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 02:47:33.232427 4956 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:47:33.602401 4961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:33.602797 4961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:33.602982 4961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:33.611253 4961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:33.611444 4961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:33.611644 4961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:47:33.622043 4961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:35539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=32875
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:33.623562 4961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:33.624982 4961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:33.639544 4967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:33.639613 4969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:33.640095 4972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:33.641098 4961 server_base.cc:1034] running on GCE node
I20260812 02:47:33.642105 4961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:33.643380 4961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:33.644665 4961 hybrid_clock.cc:648] HybridClock initialized: now 1786502853644620 us; error 59 us; skew 500 ppm
I20260812 02:47:33.645057 4961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:33 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502853, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:33.648649 4961 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:47:33.650382 4961 webserver.cc:460] Webserver started at http://127.17.196.194:32875/ using document root <none> and password file <none>
I20260812 02:47:33.651139 4961 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:33.651280 4961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:33.657230 4961 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 02:47:33.660439 4979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:33.662411 4961 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:47:33.662662 4961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "9945b3ce8bcb4c1ca9d4ee34c318cf8c"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "91e7f512c4f071f416c380b0d1cc46a2"
server_key_iv: "64eb395e699fbbc8483f447ea7bf65ad"
server_key_version: "encryptionkey@0"
I20260812 02:47:33.663820 4961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:33.707803 4961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:33.712333 4961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:33.712669 4961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:33.713845 4961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:33.715332 4961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:33.715435 4961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:33.715576 4961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:33.715658 4961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:33.752902 4961 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:35539
I20260812 02:47:33.753011 5121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:35539 every 8 connection(s)
I20260812 02:47:33.754590 4961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:47:33.757957 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 4961
I20260812 02:47:33.758898 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:46711
--local_ip_for_outbound_sockets=127.17.196.195
--tserver_master_addrs=127.17.196.254:40393
--webserver_port=39767
--webserver_interface=127.17.196.195
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:47:33.761416 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:33.758870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:40857)
0812 02:47:33.759923 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:33.760271 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":404,"threads_started":1}
W20260812 02:47:33.762570 5123 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:40393 (0 consecutive failures): Network error: Failed to ping master at 127.17.196.254:40393: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
I20260812 02:47:33.763814 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:33.763293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:44275)
0812 02:47:33.763528 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:33.763639 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 02:47:33.765745 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:33.764438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:60043)
0812 02:47:33.765178 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:33.765392 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":306,"threads_started":1}
W20260812 02:47:33.766100 5123 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:47:34.152771 5129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:34.153119 5129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:34.153297 5129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:34.162887 5129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:34.163107 5129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:34.163334 5129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:47:34.174603 5129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:46711
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=39767
--tserver_master_addrs=127.17.196.254:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:34.176007 5129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:34.177407 5129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:34.192099 5137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:34.192070 5141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:34.192148 5138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:34.199097 5129 server_base.cc:1034] running on GCE node
I20260812 02:47:34.199944 5129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:34.201174 5129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:34.202450 5129 hybrid_clock.cc:648] HybridClock initialized: now 1786502854202382 us; error 92 us; skew 500 ppm
I20260812 02:47:34.202881 5129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:34 dist-test-slave-ss4n krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:34.206694 5129 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:47:34.208535 5129 webserver.cc:460] Webserver started at http://127.17.196.195:39767/ using document root <none> and password file <none>
I20260812 02:47:34.209329 5129 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:34.209546 5129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:34.216180 5129 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:47:34.219743 5150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:34.221684 5129 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:47:34.221896 5129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "23970e9a1a364fe89aad47d5ea5e32de"
format_stamp: "Formatted at 2026-08-12 02:46:47 on dist-test-slave-ss4n"
server_key: "e982d7e570e746332b5ed8a405e532a2"
server_key_iv: "760aeac5f3f2bb8d7ae13c95a479b2ea"
server_key_version: "encryptionkey@0"
I20260812 02:47:34.223091 5129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:34.234539 5155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:34.233353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:46331)
0812 02:47:34.234105 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:34.234225 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":279,"threads_started":1}
I20260812 02:47:34.261833 5129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:34.267386 5129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:34.267752 5129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:34.268929 5129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:34.270499 5129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:34.270602 5129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:34.270684 5129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:34.270747 5129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:34.311357 5129 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:46711
I20260812 02:47:34.311718 5289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:46711 every 8 connection(s)
I20260812 02:47:34.313234 5129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:47:34.317763 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 5129
I20260812 02:47:34.319440 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:34.316825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:34451)
0812 02:47:34.317905 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:34.318319 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":387,"threads_started":1}
W20260812 02:47:34.320197 5290 heartbeater.cc:644] Failed to heartbeat to 127.17.196.254:40393 (0 consecutive failures): Network error: Failed to ping master at 127.17.196.254:40393: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
I20260812 02:47:34.321197 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:34.320782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:49995)
0812 02:47:34.320969 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:34.321057 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 02:47:34.322268 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:34.321845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:60913)
0812 02:47:34.322008 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:34.322132 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 02:47:34.322579 5290 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 02:47:34.767648 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:34.766768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:35747)
0812 02:47:34.767304 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:34.767403 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":260,"threads_started":1}
I20260812 02:47:35.236725 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:35.235738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:42555)
0812 02:47:35.236357 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:35.236475 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":276,"threads_started":1}
I20260812 02:47:35.324432 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:35.323312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:50957)
0812 02:47:35.323993 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:35.324138 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":288,"threads_started":1}
I20260812 02:47:35.769795 5299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:35.768932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:50577)
0812 02:47:35.769419 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:35.769624 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":256,"threads_started":1}
I20260812 02:47:36.238617 5300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:36.237793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:41273)
0812 02:47:36.238309 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:36.238402 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":272,"threads_started":1}
I20260812 02:47:36.326622 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:36.325706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:34405)
0812 02:47:36.326340 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:36.326437 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":343,"threads_started":1}
I20260812 02:47:36.771667 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:36.770843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:46395)
0812 02:47:36.771345 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:36.771440 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":258,"threads_started":1}
I20260812 02:47:37.240641 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:37.239781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:54603)
0812 02:47:37.240340 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:37.240441 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":308,"threads_started":1}
I20260812 02:47:37.329017 5304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:37.327815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:43933)
0812 02:47:37.328675 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:37.328785 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":374,"threads_started":1}
I20260812 02:47:37.773696 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:37.772766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:47875)
0812 02:47:37.773307 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:37.773414 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":286,"threads_started":1}
I20260812 02:47:38.243120 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:38.242016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:43599)
0812 02:47:38.242690 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:38.242854 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":345,"threads_started":1}
I20260812 02:47:38.331435 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:38.330253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:48759)
0812 02:47:38.331022 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:38.331134 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":454,"threads_started":1}
I20260812 02:47:38.775920 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:38.774970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:41179)
0812 02:47:38.775606 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:38.775713 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":328,"threads_started":1}
I20260812 02:47:39.245203 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:39.244294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:48125)
0812 02:47:39.244902 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:39.245012 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":335,"threads_started":1}
I20260812 02:47:39.333428 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:39.332579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:45005)
0812 02:47:39.333140 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:39.333246 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":289,"threads_started":1}
I20260812 02:47:39.778091 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:39.777139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:40507)
0812 02:47:39.777794 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:39.777903 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":277,"threads_started":1}
I20260812 02:47:40.247762 5314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:40.246708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:60093)
0812 02:47:40.247406 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:40.247521 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":330,"threads_started":1}
I20260812 02:47:40.335537 5315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:40.334659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:58987)
0812 02:47:40.335239 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:40.335355 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":315,"threads_started":1}
I20260812 02:47:40.780512 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:40.779314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:53841)
0812 02:47:40.780111 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:40.780239 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":395,"threads_started":1}
I20260812 02:47:41.249943 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:41.248940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:38989)
0812 02:47:41.249659 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:41.249759 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":406,"threads_started":1}
I20260812 02:47:41.337446 5318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:41.336568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:54201)
0812 02:47:41.337123 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:41.337220 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":232,"threads_started":1}
I20260812 02:47:41.782769 5319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:41.781860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:34375)
0812 02:47:41.782460 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:41.782565 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":318,"threads_started":1}
I20260812 02:47:42.252032 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:42.251053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:42659)
0812 02:47:42.251670 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:42.251799 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":288,"threads_started":1}
I20260812 02:47:42.339496 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:42.338671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:41735)
0812 02:47:42.339217 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:42.339312 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":300,"threads_started":1}
I20260812 02:47:42.784943 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:42.784020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:56759)
0812 02:47:42.784647 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:42.784762 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":286,"threads_started":1}
I20260812 02:47:43.254065 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:43.253152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:46383)
0812 02:47:43.253797 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:43.253886 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":401,"threads_started":1}
I20260812 02:47:43.341792 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:43.340669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:48791)
0812 02:47:43.341345 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:43.341469 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":345,"threads_started":1}
I20260812 02:47:43.787130 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:43.786284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:41243)
0812 02:47:43.786839 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:43.786945 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":310,"threads_started":1}
I20260812 02:47:44.256384 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:44.255310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:36253)
0812 02:47:44.256050 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:44.256178 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":368,"threads_started":1}
I20260812 02:47:44.343951 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:44.343022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:36117)
0812 02:47:44.343647 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:44.343763 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":300,"threads_started":1}
I20260812 02:47:44.789913 5331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:44.788392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:35311)
0812 02:47:44.789338 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:44.789474 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":533,"threads_started":1}
I20260812 02:47:45.258621 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:45.257683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:40515)
0812 02:47:45.258250 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:45.258360 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":285,"threads_started":1}
I20260812 02:47:45.346266 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:45.345210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:56161)
0812 02:47:45.345859 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:45.345965 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":415,"threads_started":1}
I20260812 02:47:45.792135 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:45.791245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:55139)
0812 02:47:45.791838 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:45.791941 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":345,"threads_started":1}
I20260812 02:47:46.260771 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:46.259882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:42471)
0812 02:47:46.260456 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:46.260566 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":282,"threads_started":1}
I20260812 02:47:46.348692 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:46.347595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:42377)
0812 02:47:46.348188 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:46.348337 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":279,"threads_started":1}
I20260812 02:47:46.794471 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:46.793521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:49501)
0812 02:47:46.794167 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:46.794267 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":326,"threads_started":1}
I20260812 02:47:47.263307 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:47.262210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:50783)
0812 02:47:47.262907 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:47.263033 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":420,"threads_started":1}
I20260812 02:47:47.350759 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:47.349821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:60239)
0812 02:47:47.350429 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:47.350548 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":304,"threads_started":1}
I20260812 02:47:47.796876 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:47.795878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:38331)
0812 02:47:47.796516 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:47.796664 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":372,"threads_started":1}
I20260812 02:47:48.265676 5342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:48.264710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:60843)
0812 02:47:48.265259 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:48.265371 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":293,"threads_started":1}
I20260812 02:47:48.352792 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:48.351885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:49089)
0812 02:47:48.352445 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:48.352538 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":305,"threads_started":1}
I20260812 02:47:48.799435 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:48.798536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.194:49079)
0812 02:47:48.799098 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:48.799224 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":282,"threads_started":1}
I20260812 02:47:49.267830 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:49.266876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.193:53047)
0812 02:47:49.267549 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:49.267656 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":296,"threads_started":1}
I20260812 02:47:49.321760 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 4756
I20260812 02:47:49.345570 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:49.323282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48530 (local address 127.17.196.193:41523)
0812 02:47:49.324133 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:49.324155 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:49.324498 (+ 343us) server_negotiation.cc:407] Connection header received
0812 02:47:49.326839 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:49.326851 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:49.327494 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:49.327872 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:49.330219 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:49.331750 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:49.333333 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:49.334212 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:49.334405 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:49.337803 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:49.337812 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:49.337832 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:49.337902 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:49.341202 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:49.342333 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:49.342342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:49.342350 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:49.342551 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:49.342983 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:49.342987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:49.342989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:49.343710 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:49.343956 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:49.344636 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 02:47:49.345193 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":407,"threads_started":1}
I20260812 02:47:49.347249 4948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:49.355202 5352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:49.354341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:40393 (local address 127.17.196.195:49751)
0812 02:47:49.354881 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:49.355001 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.254:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":285,"threads_started":1}
I20260812 02:47:49.442107 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 4756
I20260812 02:47:49.467342 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 4961
I20260812 02:47:49.489421 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:49.468675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.17.196.194:35539)
0812 02:47:49.469565 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:49.469587 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:49.469639 (+ 52us) server_negotiation.cc:407] Connection header received
0812 02:47:49.471783 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:49.471796 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:49.472334 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:49.472658 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:49.474459 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:49.475765 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:49.477306 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:49.478094 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:49.478284 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:49.481403 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:49.481412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:49.481442 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:49.481559 (+ 117us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:49.485016 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:49.486135 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:49.486143 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:49.486150 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:49.486338 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:49.486893 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:49.486907 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:49.486911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:49.487636 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:49.487918 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:49.488503 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 02:47:49.489075 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":483,"threads_started":1}
I20260812 02:47:49.491283 5117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:49.582238 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 4961
I20260812 02:47:49.605952 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 5129
I20260812 02:47:49.627661 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:49.607208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.17.196.195:46711)
0812 02:47:49.607934 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:49.607949 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:49.607995 (+ 46us) server_negotiation.cc:407] Connection header received
0812 02:47:49.610106 (+ 2111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:49.610119 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:49.610611 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:49.611016 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:49.612634 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:49.614031 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:49.615400 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:49.616148 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:49.616364 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:49.619670 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:49.619679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:49.619698 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:49.619758 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:49.623303 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:49.624183 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:49.624192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:49.624201 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:49.624418 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:49.624915 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:49.624920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:49.624924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:49.625683 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:49.626084 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:49.626671 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 02:47:49.627295 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":347,"threads_started":1}
I20260812 02:47:49.629594 5285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:49.716038 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 5129
2026-08-12T02:47:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65800 ms)
[----------] 30 tests from SecurityITest (322097 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:47:49 dist-test-slave-ss4n krb5kdc[5360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:47:49 dist-test-slave-ss4n krb5kdc[5360](info): set up 2 sockets
Aug 12 02:47:49 dist-test-slave-ss4n krb5kdc[5360](info): commencing operation
krb5kdc: starting...
W20260812 02:47:50.984247 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:47:51 dist-test-slave-ss4n krb5kdc[5360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:47:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:47:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:47:51.336253 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46123
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:46123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:47:51.707063 5374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:51.707515 5374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:51.707718 5374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:51.716537 5374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:47:51.716739 5374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:51.716869 5374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:51.716998 5374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:47:51.717103 5374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:47:51.727783 5374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:46123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:51.729333 5374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:51.730955 5374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:51.745988 5381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:51.745954 5379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:51.746075 5384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:51.746641 5374 server_base.cc:1034] running on GCE node
I20260812 02:47:51.747525 5374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:51.749138 5374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:51.750394 5374 hybrid_clock.cc:648] HybridClock initialized: now 1786502871750310 us; error 89 us; skew 500 ppm
I20260812 02:47:51.750874 5374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:51 dist-test-slave-ss4n krb5kdc[5360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502871, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:51.760962 5374 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:47:51.763196 5374 webserver.cc:460] Webserver started at http://127.17.196.254:39367/ using document root <none> and password file <none>
I20260812 02:47:51.764021 5374 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:51.764196 5374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:51.764539 5374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:47:51.768002 5374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee"
format_stamp: "Formatted at 2026-08-12 02:47:51 on dist-test-slave-ss4n"
server_key: "a3718627517fd9a3782b984b93fa16e7"
server_key_iv: "85f60c6b0a2ef7b6a64a41e235342c03"
server_key_version: "encryptionkey@0"
I20260812 02:47:51.768901 5374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee"
format_stamp: "Formatted at 2026-08-12 02:47:51 on dist-test-slave-ss4n"
server_key: "a3718627517fd9a3782b984b93fa16e7"
server_key_iv: "85f60c6b0a2ef7b6a64a41e235342c03"
server_key_version: "encryptionkey@0"
I20260812 02:47:51.775410 5374 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260812 02:47:51.779564 5391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:51.781128 5374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:47:51.781356 5374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee"
format_stamp: "Formatted at 2026-08-12 02:47:51 on dist-test-slave-ss4n"
server_key: "a3718627517fd9a3782b984b93fa16e7"
server_key_iv: "85f60c6b0a2ef7b6a64a41e235342c03"
server_key_version: "encryptionkey@0"
I20260812 02:47:51.781685 5374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:51.808385 5374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:51.813701 5374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:51.814049 5374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:51.836459 5374 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:46123
I20260812 02:47:51.836467 5454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:46123 every 8 connection(s)
I20260812 02:47:51.838266 5374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:47:51.842254 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 5374
I20260812 02:47:51.842520 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:47:51.842561 5455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:51.843278 18195 external_mini_cluster.cc:1428] Setting key 895bac0d7b55f3895201b261b9d03ccd
I20260812 02:47:51.857183 5455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee: Bootstrap starting.
Aug 12 02:47:51 dist-test-slave-ss4n krb5kdc[5360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:47:51.860929 5455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:51.862556 5455 log.cc:824] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:51.866139 5455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee: No bootstrap required, opened a new log
I20260812 02:47:51.870687 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:51.845791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56596 (local address 127.17.196.254:46123)
0812 02:47:51.846687 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:51.846703 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:51.846758 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:47:51.849813 (+ 3055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:51.849848 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:51.850498 (+ 650us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:51.851010 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:51.852744 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:51.854431 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:51.856010 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:51.856781 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:51.857020 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:51.862343 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:51.862353 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:51.862370 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:51.862421 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:51.866030 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:51.866877 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:51.866885 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:51.866891 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:51.867082 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:51.867534 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:51.867541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:51.867549 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:51.868156 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:51.868518 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:51.869013 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 02:47:51.869544 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":396,"threads_started":1}
I20260812 02:47:51.874505 5455 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46123 } }
I20260812 02:47:51.874923 5455 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:51.875043 5455 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4d84ffb9afe407a86e42aa9b0dbc6ee, State: Initialized, Role: FOLLOWER
I20260812 02:47:51.875603 5455 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46123 } }
I20260812 02:47:51.875871 5455 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:47:51.876017 5455 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:47:51.876199 5455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:51.878814 5455 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46123 } }
I20260812 02:47:51.879331 5455 leader_election.cc:302] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4d84ffb9afe407a86e42aa9b0dbc6ee; no voters:
I20260812 02:47:51.879998 5455 leader_election.cc:288] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:47:51.880191 5460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:47:51.881457 5460 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [term 1 LEADER]: Becoming Leader. State: Replica: e4d84ffb9afe407a86e42aa9b0dbc6ee, State: Running, Role: LEADER
I20260812 02:47:51.882052 5460 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46123 } }
I20260812 02:47:51.882483 5455 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:47:51.885138 5461 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46123 } } }
I20260812 02:47:51.885174 5463 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4d84ffb9afe407a86e42aa9b0dbc6ee. Latest consensus state: current_term: 1 leader_uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d84ffb9afe407a86e42aa9b0dbc6ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46123 } } }
I20260812 02:47:51.885666 5461 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [sys.catalog]: This master's current role is: LEADER
I20260812 02:47:51.885661 5463 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee [sys.catalog]: This master's current role is: LEADER
I20260812 02:47:51.886713 5470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:47:51.894248 5470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:47:51.908154 5470 catalog_manager.cc:1347] Generated new cluster ID: 75b5851122cd4034bfd51732e833d0e3
I20260812 02:47:51.908388 5470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:47:51.923049 5470 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:47:51.924517 5470 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:47:51.944573 5470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee: Generated new TSK 0
I20260812 02:47:51.945600 5470 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:47:52.033531 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46123
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:47:52.399853 5486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:52.400218 5486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:52.400430 5486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:52.407965 5486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:52.408144 5486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:52.408350 5486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:47:52.419274 5486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:52.420713 5486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:52.422202 5486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:52.436304 5492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:52.436400 5495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:52.436426 5493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:52.437793 5486 server_base.cc:1034] running on GCE node
I20260812 02:47:52.438680 5486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:52.439793 5486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:52.441047 5486 hybrid_clock.cc:648] HybridClock initialized: now 1786502872440987 us; error 44 us; skew 500 ppm
I20260812 02:47:52.441560 5486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:52 dist-test-slave-ss4n krb5kdc[5360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502872, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:52.445144 5486 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:47:52.446851 5486 webserver.cc:460] Webserver started at http://127.17.196.193:39591/ using document root <none> and password file <none>
I20260812 02:47:52.447607 5486 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:52.447739 5486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:52.448079 5486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:47:52.451165 5486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "817917b2113e4e839a569c7ae23a9a6e"
format_stamp: "Formatted at 2026-08-12 02:47:52 on dist-test-slave-ss4n"
server_key: "ca372de4f689a2fd974aa09086416331"
server_key_iv: "8499bf482b5513d46e6faa6830ff0239"
server_key_version: "encryptionkey@0"
I20260812 02:47:52.451998 5486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "817917b2113e4e839a569c7ae23a9a6e"
format_stamp: "Formatted at 2026-08-12 02:47:52 on dist-test-slave-ss4n"
server_key: "ca372de4f689a2fd974aa09086416331"
server_key_iv: "8499bf482b5513d46e6faa6830ff0239"
server_key_version: "encryptionkey@0"
I20260812 02:47:52.457659 5486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:47:52.461515 5505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:52.463090 5486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:47:52.463299 5486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "817917b2113e4e839a569c7ae23a9a6e"
format_stamp: "Formatted at 2026-08-12 02:47:52 on dist-test-slave-ss4n"
server_key: "ca372de4f689a2fd974aa09086416331"
server_key_iv: "8499bf482b5513d46e6faa6830ff0239"
server_key_version: "encryptionkey@0"
I20260812 02:47:52.463531 5486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:52.483911 5486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:52.489315 5486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:52.489796 5486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:52.490974 5486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:52.492594 5486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:52.492717 5486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:52.492897 5486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:52.492988 5486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:52.533492 5486 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:40111
I20260812 02:47:52.533728 5638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:40111 every 8 connection(s)
I20260812 02:47:52.534977 5486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:47:52.539970 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 5486
I20260812 02:47:52.540223 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:47:52.540980 18195 external_mini_cluster.cc:1428] Setting key e01d07cedca388d7bd608abaac6b491b
Aug 12 02:47:52 dist-test-slave-ss4n krb5kdc[5360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502872, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:47:52.560127 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:52.538060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:58901 (local address 127.17.196.254:46123)
0812 02:47:52.538614 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:52.538619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:52.539964 (+ 1345us) server_negotiation.cc:407] Connection header received
0812 02:47:52.542381 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:52.542389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:52.542528 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:52.542671 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:52.544508 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:52.545339 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:52.547314 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:52.547801 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:52.547980 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:52.552763 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:52.552770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:52.552773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:52.552818 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:52.556323 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:52.557293 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:52.557298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:52.557301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:52.557444 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:52.558039 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:52.558045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:52.558049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:52.558370 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:52.558564 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:52.559414 (+ 850us) server_negotiation.cc:299] Negotiation successful
0812 02:47:52.559689 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":257,"threads_started":1}
I20260812 02:47:52.561252 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:52.538688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46123 (local address 127.17.196.193:58901)
0812 02:47:52.539734 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:52.539794 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:52.542034 (+ 2240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:52.542981 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:52.542992 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:52.543613 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:52.544289 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:52.544311 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:52.545591 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:52.545596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:52.547099 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:52.547118 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:52.548148 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:52.548170 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:52.548456 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:52.549362 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:47:52.549390 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:52.552413 (+ 3023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:52.556531 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:52.556543 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:52.556566 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:52.557114 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:52.557639 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:52.557645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:52.557649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:52.557888 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:52.558530 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:52.558540 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:52.558963 (+ 423us) client_negotiation.cc:769] Sending connection context
0812 02:47:52.559282 (+ 319us) client_negotiation.cc:240] Negotiation successful
0812 02:47:52.559905 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":397,"threads_started":1}
I20260812 02:47:52.563313 5639 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46123
I20260812 02:47:52.563753 5639 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:52.564822 5639 heartbeater.cc:511] Master 127.17.196.254:46123 requested a full tablet report, sending...
I20260812 02:47:52.567850 5411 ts_manager.cc:194] Registered new tserver with Master: 817917b2113e4e839a569c7ae23a9a6e (127.17.196.193:40111)
I20260812 02:47:52.570374 5411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:58901
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:47:52.627203 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46123
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:47:53.002092 5646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:53.002445 5646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:53.002615 5646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:53.010499 5646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:53.010630 5646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:53.010814 5646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:47:53.021421 5646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:53.022951 5646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:53.024408 5646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:53.039335 5651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:53.039372 5655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:53.040063 5652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:53.043550 5646 server_base.cc:1034] running on GCE node
I20260812 02:47:53.044386 5646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:53.045833 5646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:53.047173 5646 hybrid_clock.cc:648] HybridClock initialized: now 1786502873047110 us; error 57 us; skew 500 ppm
I20260812 02:47:53.047598 5646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:53 dist-test-slave-ss4n krb5kdc[5360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502873, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:53.051544 5646 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:47:53.053390 5646 webserver.cc:460] Webserver started at http://127.17.196.194:39797/ using document root <none> and password file <none>
I20260812 02:47:53.054296 5646 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:53.054482 5646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:53.054864 5646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:47:53.058355 5646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "a1172c3aae70469188fa1df1fa53bfb5"
format_stamp: "Formatted at 2026-08-12 02:47:53 on dist-test-slave-ss4n"
server_key: "25b00d195d384d9b8597f75b8363d161"
server_key_iv: "b5e9cc83334645ab9ae54bcf39c558c1"
server_key_version: "encryptionkey@0"
I20260812 02:47:53.059295 5646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "a1172c3aae70469188fa1df1fa53bfb5"
format_stamp: "Formatted at 2026-08-12 02:47:53 on dist-test-slave-ss4n"
server_key: "25b00d195d384d9b8597f75b8363d161"
server_key_iv: "b5e9cc83334645ab9ae54bcf39c558c1"
server_key_version: "encryptionkey@0"
I20260812 02:47:53.065856 5646 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:47:53.070225 5666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:53.071867 5646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:47:53.072147 5646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "a1172c3aae70469188fa1df1fa53bfb5"
format_stamp: "Formatted at 2026-08-12 02:47:53 on dist-test-slave-ss4n"
server_key: "25b00d195d384d9b8597f75b8363d161"
server_key_iv: "b5e9cc83334645ab9ae54bcf39c558c1"
server_key_version: "encryptionkey@0"
I20260812 02:47:53.072423 5646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:53.110095 5646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:53.115729 5646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:53.116109 5646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:53.117249 5646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:53.118809 5646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:53.118929 5646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:53.119103 5646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:53.119197 5646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:53.158030 5646 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:40899
I20260812 02:47:53.158072 5808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:40899 every 8 connection(s)
I20260812 02:47:53.159528 5646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:47:53.163419 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 5646
I20260812 02:47:53.163602 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:47:53.164072 18195 external_mini_cluster.cc:1428] Setting key 0f9a2733771267b1afbddd71a949fb4b
Aug 12 02:47:53 dist-test-slave-ss4n krb5kdc[5360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502873, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:47:53.183849 5811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.162590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:54675 (local address 127.17.196.254:46123)
0812 02:47:53.163163 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:53.163172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:53.164447 (+ 1275us) server_negotiation.cc:407] Connection header received
0812 02:47:53.166643 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:53.166648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:53.166764 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:53.166888 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:53.168948 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.170115 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.171792 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.172532 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.172695 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.176276 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:53.176286 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:53.176293 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:53.176342 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:53.179990 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:53.181117 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:53.181120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:53.181123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:53.181251 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:53.181825 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:53.181828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:53.181830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:53.182134 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:53.182260 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:53.183277 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0812 02:47:53.183465 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":271,"threads_started":1}
I20260812 02:47:53.185431 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.163253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46123 (local address 127.17.196.194:54675)
0812 02:47:53.164256 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:53.164315 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:53.166356 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:53.167219 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:53.167228 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:53.168038 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:53.168762 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.168783 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.170309 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.170313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.171628 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.171642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.172755 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.172760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.172966 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.173738 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:47:53.173763 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:53.176008 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:53.180159 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:53.180172 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:53.180193 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:53.180987 (+ 794us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:53.181368 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:53.181376 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:53.181383 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:53.181705 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:53.182273 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:53.182283 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:53.182809 (+ 526us) client_negotiation.cc:769] Sending connection context
0812 02:47:53.183132 (+ 323us) client_negotiation.cc:240] Negotiation successful
0812 02:47:53.183824 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":500,"threads_started":1}
I20260812 02:47:53.187175 5809 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46123
I20260812 02:47:53.187502 5809 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:53.188275 5809 heartbeater.cc:511] Master 127.17.196.254:46123 requested a full tablet report, sending...
I20260812 02:47:53.189894 5411 ts_manager.cc:194] Registered new tserver with Master: a1172c3aae70469188fa1df1fa53bfb5 (127.17.196.194:40899)
I20260812 02:47:53.191241 5411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:54675
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:47:53.242633 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46123
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:47:53.575178 5639 heartbeater.cc:503] Master 127.17.196.254:46123 was elected leader, sending a full tablet report...
W20260812 02:47:53.623243 5816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:53.623646 5816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:53.623842 5816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:53.631374 5816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:53.631582 5816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:53.631788 5816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:47:53.642609 5816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:53.644119 5816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:53.645635 5816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:53.660058 5823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:53.660272 5826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:53.660184 5824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:53.661453 5816 server_base.cc:1034] running on GCE node
I20260812 02:47:53.662359 5816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:53.663596 5816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:53.664849 5816 hybrid_clock.cc:648] HybridClock initialized: now 1786502873664783 us; error 51 us; skew 500 ppm
I20260812 02:47:53.665298 5816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:47:53 dist-test-slave-ss4n krb5kdc[5360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502873, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:47:53.669973 5816 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:47:53.671931 5816 webserver.cc:460] Webserver started at http://127.17.196.195:44493/ using document root <none> and password file <none>
I20260812 02:47:53.672855 5816 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:53.673125 5816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:53.673633 5816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:47:53.677158 5816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "17b771f61ee346ba8a43b52f79dcb84f"
format_stamp: "Formatted at 2026-08-12 02:47:53 on dist-test-slave-ss4n"
server_key: "94aa9b7e56e239af28d5c420850e4e0d"
server_key_iv: "b41974a0be006e7ccf6bc2ef49c743c5"
server_key_version: "encryptionkey@0"
I20260812 02:47:53.678279 5816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "17b771f61ee346ba8a43b52f79dcb84f"
format_stamp: "Formatted at 2026-08-12 02:47:53 on dist-test-slave-ss4n"
server_key: "94aa9b7e56e239af28d5c420850e4e0d"
server_key_iv: "b41974a0be006e7ccf6bc2ef49c743c5"
server_key_version: "encryptionkey@0"
I20260812 02:47:53.684679 5816 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:47:53.689029 5836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:53.690675 5816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:47:53.690898 5816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "17b771f61ee346ba8a43b52f79dcb84f"
format_stamp: "Formatted at 2026-08-12 02:47:53 on dist-test-slave-ss4n"
server_key: "94aa9b7e56e239af28d5c420850e4e0d"
server_key_iv: "b41974a0be006e7ccf6bc2ef49c743c5"
server_key_version: "encryptionkey@0"
I20260812 02:47:53.691171 5816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:53.711593 5816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:53.717096 5816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:53.717468 5816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:53.718804 5816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:53.720386 5816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:53.720498 5816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:53.720670 5816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:53.720747 5816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:53.760761 5816 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:41129
I20260812 02:47:53.760864 5971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:41129 every 8 connection(s)
I20260812 02:47:53.762455 5816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:47:53.769896 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 5816
I20260812 02:47:53.770115 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:47:53.770651 18195 external_mini_cluster.cc:1428] Setting key be80b1547cc8138502ffee0aaf246427
Aug 12 02:47:53 dist-test-slave-ss4n krb5kdc[5360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502873, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:47:53.786288 5974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.765672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:52501 (local address 127.17.196.254:46123)
0812 02:47:53.766197 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:53.766202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:53.767505 (+ 1303us) server_negotiation.cc:407] Connection header received
0812 02:47:53.769477 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:53.769516 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:53.769620 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:53.769757 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:53.771676 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.772594 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.774541 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.774963 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.775110 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.779175 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:53.779181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:53.779184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:53.779222 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:53.782451 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:53.783423 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:53.783436 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:53.783438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:53.783592 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:53.784109 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:53.784114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:53.784116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:53.784419 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:53.784609 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:53.785621 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0812 02:47:53.785875 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":272,"threads_started":1}
I20260812 02:47:53.787334 5975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.766320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:46123 (local address 127.17.196.195:52501)
0812 02:47:53.767302 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:53.767351 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:53.769158 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:53.770076 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:53.770090 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:53.770807 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:53.771462 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.771481 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.772798 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.772802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.774385 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.774400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.775237 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.775245 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.775500 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.776311 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:47:53.776341 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:53.778911 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:53.782640 (+ 3729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:53.782658 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:53.782674 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:53.783215 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:53.783724 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:47:53.783730 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:47:53.783734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:47:53.783948 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:47:53.784562 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:53.784572 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:53.785045 (+ 473us) client_negotiation.cc:769] Sending connection context
0812 02:47:53.785453 (+ 408us) client_negotiation.cc:240] Negotiation successful
0812 02:47:53.786091 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":339,"threads_started":1}
I20260812 02:47:53.788825 5972 heartbeater.cc:347] Connected to a master server at 127.17.196.254:46123
I20260812 02:47:53.789192 5972 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:53.790071 5972 heartbeater.cc:511] Master 127.17.196.254:46123 requested a full tablet report, sending...
I20260812 02:47:53.791931 5411 ts_manager.cc:194] Registered new tserver with Master: 17b771f61ee346ba8a43b52f79dcb84f (127.17.196.195:41129)
I20260812 02:47:53.792873 5411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:52501
I20260812 02:47:53.800454 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:47:53.820513 5974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.805823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52466 (local address 127.17.196.254:46123)
0812 02:47:53.806027 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:53.806032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:53.806523 (+ 491us) server_negotiation.cc:407] Connection header received
0812 02:47:53.806644 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:53.806648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:53.806720 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:53.806816 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:53.808449 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.809299 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.810826 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.811327 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.811448 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.814652 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:53.814657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:53.814661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:53.814710 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:53.817685 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:53.818451 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:53.818455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:53.818458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:53.818593 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:53.818995 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:53.818998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:53.819000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:53.819350 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:53.819551 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:53.819877 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 02:47:53.820108 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:47:53.826011 5411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:47:53.835610 5411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:47:53.863497 5987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.850864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40111 (local address 127.0.0.1:43728)
0812 02:47:53.852352 (+ 1488us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:53.852411 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:53.852694 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:53.853833 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:53.853840 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:53.853878 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:53.854987 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.855001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.857247 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.857254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.860215 (+ 2961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.860232 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.862395 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.862402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.862624 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.862690 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 02:47:53.862809 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 02:47:53.863012 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"mutex_wait_us":287,"spinlock_wait_cycles":12032,"thread_start_us":1049,"threads_started":1}
I20260812 02:47:53.864749 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.851377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:40899 (local address 127.0.0.1:36626)
0812 02:47:53.852461 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:53.852484 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:53.852695 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:53.853996 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:53.854004 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:53.854035 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:53.854639 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.854659 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.857250 (+ 2591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.857256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.860340 (+ 3084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.860357 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.863773 (+ 3416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.863782 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.863960 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.864003 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 02:47:53.864106 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 02:47:53.864255 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"mutex_wait_us":118,"spinlock_wait_cycles":2944,"thread_start_us":823,"threads_started":1}
I20260812 02:47:53.864853 5992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.852798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:41129 (local address 127.0.0.1:57828)
0812 02:47:53.856169 (+ 3371us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:53.856189 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:53.856346 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:53.856955 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:53.856960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:53.856986 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:53.857540 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.857715 (+ 175us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.859947 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.859953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.861919 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:53.861935 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.864056 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.864065 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:53.864229 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.864257 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 02:47:53.864323 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 02:47:53.864458 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3253,"mutex_wait_us":42,"thread_start_us":3063,"threads_started":1}
I20260812 02:47:53.864943 5995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.854202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.17.196.195:41129)
0812 02:47:53.855665 (+ 1463us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:53.855678 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:53.856300 (+ 622us) server_negotiation.cc:407] Connection header received
0812 02:47:53.856522 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:53.856529 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:53.856747 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:53.856954 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:53.858068 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.859755 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.862200 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.863812 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.864083 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.864308 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:53.864421 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 02:47:53.864561 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":1089,"threads_started":1}
I20260812 02:47:53.865988 5988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.851098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.17.196.193:40111)
0812 02:47:53.851919 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:53.851929 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:53.852530 (+ 601us) server_negotiation.cc:407] Connection header received
0812 02:47:53.852972 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:53.852982 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:53.853278 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:53.853459 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:53.855192 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.857046 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.860401 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.862114 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.865166 (+ 3052us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.865375 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:53.865454 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 02:47:53.865679 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":386,"threads_started":1}
I20260812 02:47:53.866160 5991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:53.851944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.17.196.194:40899)
0812 02:47:53.853322 (+ 1378us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:53.853328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:53.853351 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:47:53.853524 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:53.853532 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:53.853772 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:53.854330 (+ 558us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:53.854908 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.856977 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.860555 (+ 3578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:53.862114 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:53.862428 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:53.862643 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:53.865632 (+ 2989us) server_negotiation.cc:299] Negotiation successful
0812 02:47:53.865794 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1222,"thread_start_us":1104,"threads_started":1}
I20260812 02:47:53.869843 5892 tablet_service.cc:1467] Processing CreateTablet for tablet d2ca991728f8417eaa1d23fad1dd64c0 (DEFAULT_TABLE table=test-table [id=352db88e5b674835b23c94cc3129d80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:53.870204 5728 tablet_service.cc:1467] Processing CreateTablet for tablet d2ca991728f8417eaa1d23fad1dd64c0 (DEFAULT_TABLE table=test-table [id=352db88e5b674835b23c94cc3129d80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:53.870174 5565 tablet_service.cc:1467] Processing CreateTablet for tablet d2ca991728f8417eaa1d23fad1dd64c0 (DEFAULT_TABLE table=test-table [id=352db88e5b674835b23c94cc3129d80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:53.871567 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ca991728f8417eaa1d23fad1dd64c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:53.871601 5728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ca991728f8417eaa1d23fad1dd64c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:53.871603 5892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ca991728f8417eaa1d23fad1dd64c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:53.887879 5996 tablet_bootstrap.cc:492] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: Bootstrap starting.
I20260812 02:47:53.889966 5997 tablet_bootstrap.cc:492] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Bootstrap starting.
I20260812 02:47:53.893152 5996 tablet_bootstrap.cc:654] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:53.893589 6000 tablet_bootstrap.cc:492] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Bootstrap starting.
I20260812 02:47:53.894541 5996 log.cc:824] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:53.895313 5997 tablet_bootstrap.cc:654] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:53.896749 5997 log.cc:824] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:53.897441 6000 tablet_bootstrap.cc:654] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:53.898257 5996 tablet_bootstrap.cc:492] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: No bootstrap required, opened a new log
I20260812 02:47:53.898635 5996 ts_tablet_manager.cc:1397] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260812 02:47:53.899359 6000 log.cc:824] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:53.900635 5997 tablet_bootstrap.cc:492] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: No bootstrap required, opened a new log
I20260812 02:47:53.900990 5997 ts_tablet_manager.cc:1397] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 02:47:53.903007 6000 tablet_bootstrap.cc:492] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: No bootstrap required, opened a new log
I20260812 02:47:53.903326 6000 ts_tablet_manager.cc:1397] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.001s
I20260812 02:47:53.908245 5996 raft_consensus.cc:348] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:53.908756 5996 raft_consensus.cc:374] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:53.908872 5996 raft_consensus.cc:729] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1172c3aae70469188fa1df1fa53bfb5, State: Initialized, Role: FOLLOWER
I20260812 02:47:53.909686 5996 consensus_queue.cc:260] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:53.910998 5809 heartbeater.cc:503] Master 127.17.196.254:46123 was elected leader, sending a full tablet report...
I20260812 02:47:53.912727 5997 raft_consensus.cc:348] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:53.913245 5997 raft_consensus.cc:374] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:53.913341 5997 raft_consensus.cc:729] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17b771f61ee346ba8a43b52f79dcb84f, State: Initialized, Role: FOLLOWER
I20260812 02:47:53.914215 5997 consensus_queue.cc:260] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:53.914515 5996 ts_tablet_manager.cc:1428] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: Time spent starting tablet: real 0.016s user 0.010s sys 0.003s
I20260812 02:47:53.921710 5972 heartbeater.cc:503] Master 127.17.196.254:46123 was elected leader, sending a full tablet report...
I20260812 02:47:53.922891 5997 ts_tablet_manager.cc:1428] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Time spent starting tablet: real 0.022s user 0.015s sys 0.008s
I20260812 02:47:53.930457 6000 raft_consensus.cc:348] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:53.931115 6000 raft_consensus.cc:374] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:53.931267 6000 raft_consensus.cc:729] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817917b2113e4e839a569c7ae23a9a6e, State: Initialized, Role: FOLLOWER
I20260812 02:47:53.932058 6000 consensus_queue.cc:260] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:53.934235 6000 ts_tablet_manager.cc:1428] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Time spent starting tablet: real 0.031s user 0.014s sys 0.003s
W20260812 02:47:54.014420 5973 tablet.cc:2369] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:47:54.015417 6004 raft_consensus.cc:482] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:47:54.015805 6004 raft_consensus.cc:504] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:54.017702 6004 leader_election.cc:288] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 817917b2113e4e839a569c7ae23a9a6e (127.17.196.193:40111), 17b771f61ee346ba8a43b52f79dcb84f (127.17.196.195:41129)
I20260812 02:47:54.025888 5988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:54.018206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:56433 (local address 127.17.196.193:40111)
0812 02:47:54.018410 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:54.018415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:54.019102 (+ 687us) server_negotiation.cc:407] Connection header received
0812 02:47:54.019402 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:54.019408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:54.019496 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:54.019570 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:54.020579 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.021472 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.023868 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.024713 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.024849 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:54.024906 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:54.025423 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 02:47:54.025594 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:47:54.025990 6009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:54.019088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:41129 (local address 127.17.196.194:48513)
0812 02:47:54.019746 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:54.019761 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:54.019900 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:54.020284 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:54.020288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:54.020310 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:54.020741 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:54.020754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.021977 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.021981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:54.023680 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:54.023695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.025135 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.025140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:54.025276 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:54.025300 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:47:54.025421 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 02:47:54.025594 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":322,"threads_started":1}
I20260812 02:47:54.025995 5995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:54.019354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:48513 (local address 127.17.196.195:41129)
0812 02:47:54.019566 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:54.019572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:54.019865 (+ 293us) server_negotiation.cc:407] Connection header received
0812 02:47:54.020040 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:54.020047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:54.020130 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:54.020244 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:54.020906 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.021805 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.023918 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.024900 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.025104 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:54.025188 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:54.025544 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 02:47:54.025695 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:47:54.026108 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:54.018204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:40111 (local address 127.17.196.194:56433)
0812 02:47:54.018907 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:54.018925 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:54.019179 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:54.019780 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:54.019784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:54.019807 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:54.020302 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:54.020318 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.021686 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.021691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:54.023641 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:54.023662 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.024984 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.024996 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:54.025205 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:54.025260 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 02:47:54.025349 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 02:47:54.025615 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":379,"threads_started":1}
I20260812 02:47:54.027278 5587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ca991728f8417eaa1d23fad1dd64c0" candidate_uuid: "a1172c3aae70469188fa1df1fa53bfb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817917b2113e4e839a569c7ae23a9a6e" is_pre_election: true
I20260812 02:47:54.027318 5916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ca991728f8417eaa1d23fad1dd64c0" candidate_uuid: "a1172c3aae70469188fa1df1fa53bfb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17b771f61ee346ba8a43b52f79dcb84f" is_pre_election: true
I20260812 02:47:54.027851 5587 raft_consensus.cc:2454] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1172c3aae70469188fa1df1fa53bfb5 in term 0.
I20260812 02:47:54.027868 5916 raft_consensus.cc:2454] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1172c3aae70469188fa1df1fa53bfb5 in term 0.
I20260812 02:47:54.028687 5667 leader_election.cc:302] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817917b2113e4e839a569c7ae23a9a6e, a1172c3aae70469188fa1df1fa53bfb5; no voters:
I20260812 02:47:54.029171 6004 raft_consensus.cc:2781] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:47:54.029348 6004 raft_consensus.cc:482] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:47:54.029487 6004 raft_consensus.cc:3037] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:54.032155 6004 raft_consensus.cc:504] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:54.032878 6004 leader_election.cc:288] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [CANDIDATE]: Term 1 election: Requested vote from peers 817917b2113e4e839a569c7ae23a9a6e (127.17.196.193:40111), 17b771f61ee346ba8a43b52f79dcb84f (127.17.196.195:41129)
I20260812 02:47:54.033717 5587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ca991728f8417eaa1d23fad1dd64c0" candidate_uuid: "a1172c3aae70469188fa1df1fa53bfb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817917b2113e4e839a569c7ae23a9a6e"
I20260812 02:47:54.033753 5916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ca991728f8417eaa1d23fad1dd64c0" candidate_uuid: "a1172c3aae70469188fa1df1fa53bfb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17b771f61ee346ba8a43b52f79dcb84f"
I20260812 02:47:54.033977 5587 raft_consensus.cc:3037] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:54.033994 5916 raft_consensus.cc:3037] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:54.036489 5587 raft_consensus.cc:2454] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1172c3aae70469188fa1df1fa53bfb5 in term 1.
I20260812 02:47:54.036720 5916 raft_consensus.cc:2454] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1172c3aae70469188fa1df1fa53bfb5 in term 1.
I20260812 02:47:54.037185 5667 leader_election.cc:302] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817917b2113e4e839a569c7ae23a9a6e, a1172c3aae70469188fa1df1fa53bfb5; no voters:
I20260812 02:47:54.037731 6004 raft_consensus.cc:2781] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 1 FOLLOWER]: Leader election won for term 1
W20260812 02:47:54.037842 5640 tablet.cc:2369] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:47:54.038391 6004 raft_consensus.cc:686] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 1 LEADER]: Becoming Leader. State: Replica: a1172c3aae70469188fa1df1fa53bfb5, State: Running, Role: LEADER
I20260812 02:47:54.039017 6004 consensus_queue.cc:237] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } }
I20260812 02:47:54.044047 5410 catalog_manager.cc:5495] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1172c3aae70469188fa1df1fa53bfb5 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "a1172c3aae70469188fa1df1fa53bfb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1172c3aae70469188fa1df1fa53bfb5" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40899 } health_report { overall_health: HEALTHY } } }
I20260812 02:47:54.065606 5991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:54.059057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.17.196.194:40899)
0812 02:47:54.059312 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:54.059320 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:54.059342 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:47:54.059617 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:54.059622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:54.059722 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:54.059902 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:47:54.060322 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.061075 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.062721 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.063161 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.063314 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:54.063972 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:47:54.064777 (+ 805us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:47:54.064935 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:54.065223 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 02:47:54.065317 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 02:47:54.075137 5916 raft_consensus.cc:1261] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 1 FOLLOWER]: Refusing update from remote peer a1172c3aae70469188fa1df1fa53bfb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:47:54.075139 5587 raft_consensus.cc:1261] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 1 FOLLOWER]: Refusing update from remote peer a1172c3aae70469188fa1df1fa53bfb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:47:54.076403 6010 consensus_queue.cc:1035] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17b771f61ee346ba8a43b52f79dcb84f" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 41129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:47:54.077032 6004 consensus_queue.cc:1035] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "817917b2113e4e839a569c7ae23a9a6e" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 40111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:47:54.111344 5410 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:47:54.111687 5410 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:47:54.115528 5410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4d84ffb9afe407a86e42aa9b0dbc6ee: Sending DeleteTablet for 3 replicas of tablet d2ca991728f8417eaa1d23fad1dd64c0
I20260812 02:47:54.117039 5565 tablet_service.cc:1514] Processing DeleteTablet for tablet d2ca991728f8417eaa1d23fad1dd64c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:54 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:43728
I20260812 02:47:54.117373 5892 tablet_service.cc:1514] Processing DeleteTablet for tablet d2ca991728f8417eaa1d23fad1dd64c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:54 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:57828
I20260812 02:47:54.118201 5728 tablet_service.cc:1514] Processing DeleteTablet for tablet d2ca991728f8417eaa1d23fad1dd64c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:54 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:36626
I20260812 02:47:54.118484 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 5486
I20260812 02:47:54.131809 6026 tablet_replica.cc:330] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: stopping tablet replica
I20260812 02:47:54.132639 6026 raft_consensus.cc:2229] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:47:54.134354 6026 raft_consensus.cc:2258] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:54.134662 6024 tablet_replica.cc:330] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: stopping tablet replica
I20260812 02:47:54.135339 6024 raft_consensus.cc:2229] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:47:54.136453 6024 raft_consensus.cc:2258] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:54.137210 6023 tablet_replica.cc:330] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: stopping tablet replica
I20260812 02:47:54.137959 6023 raft_consensus.cc:2229] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:47:54.138792 6023 raft_consensus.cc:2258] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:54.139184 6026 ts_tablet_manager.cc:1905] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:47:54.143157 6024 ts_tablet_manager.cc:1905] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:47:54 dist-test-slave-ss4n krb5kdc[5360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:47:54.147532 6026 ts_tablet_manager.cc:1918] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:54.147871 6026 log.cc:1196] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/d2ca991728f8417eaa1d23fad1dd64c0
I20260812 02:47:54.148607 6023 ts_tablet_manager.cc:1905] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:47:54.148696 6026 ts_tablet_manager.cc:1939] T d2ca991728f8417eaa1d23fad1dd64c0 P a1172c3aae70469188fa1df1fa53bfb5: Deleting consensus metadata
I20260812 02:47:54.150938 5395 catalog_manager.cc:4841] TS a1172c3aae70469188fa1df1fa53bfb5 (127.17.196.194:40899): tablet d2ca991728f8417eaa1d23fad1dd64c0 (table test-table [id=352db88e5b674835b23c94cc3129d80d]) successfully deleted
I20260812 02:47:54.154808 6024 ts_tablet_manager.cc:1918] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:54.155124 6024 log.cc:1196] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/d2ca991728f8417eaa1d23fad1dd64c0
I20260812 02:47:54.155833 6024 ts_tablet_manager.cc:1939] T d2ca991728f8417eaa1d23fad1dd64c0 P 17b771f61ee346ba8a43b52f79dcb84f: Deleting consensus metadata
I20260812 02:47:54.156456 5988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:54.119839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.17.196.193:40111)
0812 02:47:54.120071 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:54.120078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:54.130575 (+ 10497us) server_negotiation.cc:407] Connection header received
0812 02:47:54.130855 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:54.130863 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:54.130971 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:54.131116 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:54.133632 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.135068 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.137895 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.138787 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.138911 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:54.146743 (+ 7832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:54.146768 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:54.146792 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:54.146872 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:54.151727 (+ 4855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:54.152991 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:54.153019 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:54.153024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:54.153256 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:54.153881 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:54.153887 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:54.153891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:54.154541 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:54.154917 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:54.155652 (+ 735us) server_negotiation.cc:299] Negotiation successful
0812 02:47:54.155954 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:47:54.157059 6023 ts_tablet_manager.cc:1918] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:54.157317 6023 log.cc:1196] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/d2ca991728f8417eaa1d23fad1dd64c0
I20260812 02:47:54.157989 6023 ts_tablet_manager.cc:1939] T d2ca991728f8417eaa1d23fad1dd64c0 P 817917b2113e4e839a569c7ae23a9a6e: Deleting consensus metadata
I20260812 02:47:54.158746 5634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:54.159142 5393 catalog_manager.cc:4841] TS 17b771f61ee346ba8a43b52f79dcb84f (127.17.196.195:41129): tablet d2ca991728f8417eaa1d23fad1dd64c0 (table test-table [id=352db88e5b674835b23c94cc3129d80d]) successfully deleted
I20260812 02:47:54.287050 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 5486
I20260812 02:47:54.287039 5393 catalog_manager.cc:4841] TS 817917b2113e4e839a569c7ae23a9a6e (127.17.196.193:40111): tablet d2ca991728f8417eaa1d23fad1dd64c0 (table test-table [id=352db88e5b674835b23c94cc3129d80d]) successfully deleted
I20260812 02:47:54.309747 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 5646
Aug 12 02:47:54 dist-test-slave-ss4n krb5kdc[5360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:47:54.330986 5991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:54.310907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.17.196.194:40899)
0812 02:47:54.311143 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:54.311149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:54.311180 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:47:54.311253 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:54.311257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:54.311325 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:54.311435 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:54.313332 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.314412 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.316376 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.316883 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.316996 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:54.322762 (+ 5766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:54.322787 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:54.322809 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:54.322869 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:54.327403 (+ 4534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:54.328381 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:54.328392 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:54.328396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:54.328621 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:54.329131 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:54.329136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:54.329140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:54.329656 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:54.330039 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:54.330297 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 02:47:54.330558 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 02:47:54.331952 5804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:54.452066 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 5646
I20260812 02:47:54.475656 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 5816
Aug 12 02:47:54 dist-test-slave-ss4n krb5kdc[5360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:47:54.494212 5995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:54.476474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57842 (local address 127.17.196.195:41129)
0812 02:47:54.476723 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:54.476728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:54.476750 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:47:54.476836 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:54.476840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:54.476924 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:54.477017 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:54.478672 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.479403 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.480811 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:54.481271 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:54.481371 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:54.486633 (+ 5262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:54.486652 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:54.486680 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:47:54.486721 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:54.491109 (+ 4388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:54.491883 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:54.491891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:54.491894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:54.492090 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:47:54.492637 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:47:54.492641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:47:54.492644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:47:54.493060 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:54.493304 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:54.493731 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 02:47:54.493912 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:47:54.494983 5967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:54.587100 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 5816
I20260812 02:47:54.609261 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 5374
I20260812 02:47:54.610723 5448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:54.726049 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 5374
2026-08-12T02:47:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5020 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T02:47:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:47:54Z Disabled control of system clock
I20260812 02:47:54.833407 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:35075
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:35075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:47:55.198609 6038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:55.198977 6038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:55.199163 6038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:55.206979 6038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:47:55.207105 6038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:55.207180 6038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:55.207270 6038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:47:55.207351 6038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:47:55.217178 6038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:35075
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:35075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:55.218628 6038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:55.220510 6038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:55.233304 6044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:55.233234 6045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:55.233281 6047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:55.234359 6038 server_base.cc:1034] running on GCE node
I20260812 02:47:55.235308 6038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:55.236949 6038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:55.238229 6038 hybrid_clock.cc:648] HybridClock initialized: now 1786502875238181 us; error 46 us; skew 500 ppm
I20260812 02:47:55.238751 6038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:47:55.241756 6038 webserver.cc:460] Webserver started at http://127.17.196.254:34097/ using document root <none> and password file <none>
I20260812 02:47:55.242666 6038 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:55.242831 6038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:55.243227 6038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:47:55.246851 6038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "89b6d4da85374170ba7fdd874d9240ee"
format_stamp: "Formatted at 2026-08-12 02:47:55 on dist-test-slave-ss4n"
server_key: "fc67570a51b9ed2a84a0f343fe20ae50"
server_key_iv: "f48cd2ee6e1fb02b33f0abd973b15007"
server_key_version: "encryptionkey@0"
I20260812 02:47:55.247844 6038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "89b6d4da85374170ba7fdd874d9240ee"
format_stamp: "Formatted at 2026-08-12 02:47:55 on dist-test-slave-ss4n"
server_key: "fc67570a51b9ed2a84a0f343fe20ae50"
server_key_iv: "f48cd2ee6e1fb02b33f0abd973b15007"
server_key_version: "encryptionkey@0"
I20260812 02:47:55.254201 6038 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260812 02:47:55.258345 6053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:55.259996 6038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:47:55.260233 6038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "89b6d4da85374170ba7fdd874d9240ee"
format_stamp: "Formatted at 2026-08-12 02:47:55 on dist-test-slave-ss4n"
server_key: "fc67570a51b9ed2a84a0f343fe20ae50"
server_key_iv: "f48cd2ee6e1fb02b33f0abd973b15007"
server_key_version: "encryptionkey@0"
I20260812 02:47:55.260510 6038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:55.290699 6038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:55.291733 6038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:55.292112 6038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:55.314110 6038 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:35075
I20260812 02:47:55.314146 6110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:35075 every 8 connection(s)
I20260812 02:47:55.315682 6038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:47:55.318668 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 6038
I20260812 02:47:55.318878 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:47:55.319484 18195 external_mini_cluster.cc:1428] Setting key d64d7d207b93c700ae8ad969d40a847a
I20260812 02:47:55.320704 6112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:55.337407 6112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee: Bootstrap starting.
I20260812 02:47:55.338728 6117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:55.321810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47592 (local address 127.17.196.254:35075)
0812 02:47:55.323065 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:55.323087 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:55.323138 (+ 51us) server_negotiation.cc:407] Connection header received
0812 02:47:55.325868 (+ 2730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:55.325899 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:55.326584 (+ 685us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:55.327040 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:55.327864 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:55.330653 (+ 2789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:55.333108 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:55.334167 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:55.334364 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:55.335878 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:55.335887 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:55.335904 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:47:55.336563 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:55.336663 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:47:55.336689 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:55.336786 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:55.337105 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 02:47:55.337659 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":489,"threads_started":1}
I20260812 02:47:55.341060 6112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:55.342357 6112 log.cc:824] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:55.345682 6112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee: No bootstrap required, opened a new log
I20260812 02:47:55.353890 6112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b6d4da85374170ba7fdd874d9240ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35075 } }
I20260812 02:47:55.354315 6112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:55.354400 6112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b6d4da85374170ba7fdd874d9240ee, State: Initialized, Role: FOLLOWER
I20260812 02:47:55.355031 6112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b6d4da85374170ba7fdd874d9240ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35075 } }
I20260812 02:47:55.355261 6112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:47:55.355368 6112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:47:55.355504 6112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:55.358052 6112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b6d4da85374170ba7fdd874d9240ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35075 } }
I20260812 02:47:55.358800 6112 leader_election.cc:302] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89b6d4da85374170ba7fdd874d9240ee; no voters:
I20260812 02:47:55.359666 6112 leader_election.cc:288] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:47:55.359956 6119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:47:55.360976 6119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [term 1 LEADER]: Becoming Leader. State: Replica: 89b6d4da85374170ba7fdd874d9240ee, State: Running, Role: LEADER
I20260812 02:47:55.361665 6119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b6d4da85374170ba7fdd874d9240ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35075 } }
I20260812 02:47:55.362432 6112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:47:55.365002 6120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89b6d4da85374170ba7fdd874d9240ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b6d4da85374170ba7fdd874d9240ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35075 } } }
I20260812 02:47:55.365149 6121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89b6d4da85374170ba7fdd874d9240ee. Latest consensus state: current_term: 1 leader_uuid: "89b6d4da85374170ba7fdd874d9240ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b6d4da85374170ba7fdd874d9240ee" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35075 } } }
I20260812 02:47:55.365592 6120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [sys.catalog]: This master's current role is: LEADER
I20260812 02:47:55.365662 6121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee [sys.catalog]: This master's current role is: LEADER
I20260812 02:47:55.366832 6128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:47:55.374619 6128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:47:55.389072 6128 catalog_manager.cc:1347] Generated new cluster ID: 503633c6c8cf43f5939c3966078a6239
I20260812 02:47:55.389232 6128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:47:55.401655 6128 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:47:55.403187 6128 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:47:55.413820 6128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee: Generated new TSK 0
I20260812 02:47:55.414928 6128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:47:55.425282 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:35075
--builtin_ntp_servers=127.17.196.212:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:47:55.804418 6144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:55.804747 6144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:55.804901 6144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:55.812773 6144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:55.812911 6144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:55.813100 6144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:47:55.823266 6144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:35075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:55.825102 6144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:55.827205 6144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:55.843214 6153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:55.843268 6150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:55.843231 6149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:55.844342 6144 server_base.cc:1034] running on GCE node
I20260812 02:47:55.845108 6144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:55.846410 6144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:55.847759 6144 hybrid_clock.cc:648] HybridClock initialized: now 1786502875847682 us; error 73 us; skew 500 ppm
I20260812 02:47:55.848173 6144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:47:55.851037 6144 webserver.cc:460] Webserver started at http://127.17.196.193:40965/ using document root <none> and password file <none>
I20260812 02:47:55.851814 6144 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:55.851963 6144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:55.852308 6144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:47:55.855571 6144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "d168bd1e95524674a072aead80b1100f"
format_stamp: "Formatted at 2026-08-12 02:47:55 on dist-test-slave-ss4n"
server_key: "cfebcd4da6e615990def6fc7747aa1a5"
server_key_iv: "bad708d7ab220e4ec59ead03f6ac4a89"
server_key_version: "encryptionkey@0"
I20260812 02:47:55.856457 6144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "d168bd1e95524674a072aead80b1100f"
format_stamp: "Formatted at 2026-08-12 02:47:55 on dist-test-slave-ss4n"
server_key: "cfebcd4da6e615990def6fc7747aa1a5"
server_key_iv: "bad708d7ab220e4ec59ead03f6ac4a89"
server_key_version: "encryptionkey@0"
I20260812 02:47:55.862959 6144 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 02:47:55.867504 6159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:55.869141 6144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 02:47:55.869365 6144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "d168bd1e95524674a072aead80b1100f"
format_stamp: "Formatted at 2026-08-12 02:47:55 on dist-test-slave-ss4n"
server_key: "cfebcd4da6e615990def6fc7747aa1a5"
server_key_iv: "bad708d7ab220e4ec59ead03f6ac4a89"
server_key_version: "encryptionkey@0"
I20260812 02:47:55.869800 6144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:55.889675 6144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:55.890607 6144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:55.890940 6144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:55.892103 6144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:55.893756 6144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:55.894022 6144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:55.894215 6144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:55.894325 6144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:55.936795 6144 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:38217
I20260812 02:47:55.936846 6289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:38217 every 8 connection(s)
I20260812 02:47:55.938491 6144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:47:55.942826 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 6144
I20260812 02:47:55.943046 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:47:55.943621 18195 external_mini_cluster.cc:1428] Setting key e5c1e7678ccc3fb327c545ed5e508b8f
I20260812 02:47:55.947350 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:35075
--builtin_ntp_servers=127.17.196.212:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:47:55.955401 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:55.941609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:52769 (local address 127.17.196.254:35075)
0812 02:47:55.942215 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:55.942220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:55.943887 (+ 1667us) server_negotiation.cc:407] Connection header received
0812 02:47:55.946200 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:55.946204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:55.946274 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:55.946368 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:55.948015 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:55.948803 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:55.951729 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:55.952208 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:55.952329 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:55.954366 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:55.954371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:55.954375 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:47:55.954399 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:55.954446 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:47:55.954459 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:55.954539 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:55.954957 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 02:47:55.955076 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":335,"threads_started":1}
I20260812 02:47:55.956976 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:55.942408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:35075 (local address 127.17.196.193:52769)
0812 02:47:55.943649 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:55.943715 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:55.945863 (+ 2148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:55.946709 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:55.946722 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:55.946859 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:55.947799 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:55.947830 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:55.949001 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:55.949010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:55.951353 (+ 2343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:55.951375 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:55.952491 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:55.952501 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:55.952891 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:55.953885 (+ 994us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:47:55.953891 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:55.953960 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:47:55.954097 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:55.954572 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:55.954587 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:55.954598 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 02:47:55.954830 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 02:47:55.955363 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":353,"threads_started":1}
I20260812 02:47:55.959151 6290 heartbeater.cc:347] Connected to a master server at 127.17.196.254:35075
I20260812 02:47:55.959539 6290 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:55.960475 6290 heartbeater.cc:511] Master 127.17.196.254:35075 requested a full tablet report, sending...
I20260812 02:47:55.963178 6072 ts_manager.cc:194] Registered new tserver with Master: d168bd1e95524674a072aead80b1100f (127.17.196.193:38217)
I20260812 02:47:55.965080 6072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.193:52769
W20260812 02:47:56.331163 6294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:56.331578 6294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:56.331713 6294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:56.341617 6294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:56.341747 6294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:56.341858 6294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:47:56.351979 6294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:35075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:56.353363 6294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:56.355332 6294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:56.370230 6302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:56.370175 6304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:56.370374 6301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:56.371227 6294 server_base.cc:1034] running on GCE node
I20260812 02:47:56.371941 6294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:56.373045 6294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:56.374313 6294 hybrid_clock.cc:648] HybridClock initialized: now 1786502876374243 us; error 60 us; skew 500 ppm
I20260812 02:47:56.374768 6294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:47:56.377696 6294 webserver.cc:460] Webserver started at http://127.17.196.194:34725/ using document root <none> and password file <none>
I20260812 02:47:56.378546 6294 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:56.378693 6294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:56.379058 6294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:47:56.382431 6294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "ad1900d96efd4cb6953558c1c3dba0ca"
format_stamp: "Formatted at 2026-08-12 02:47:56 on dist-test-slave-ss4n"
server_key: "790e93910ef917ab97c99121a858c20f"
server_key_iv: "5c7dcdb1848c38d2238af6eedebbd7f0"
server_key_version: "encryptionkey@0"
I20260812 02:47:56.383337 6294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "ad1900d96efd4cb6953558c1c3dba0ca"
format_stamp: "Formatted at 2026-08-12 02:47:56 on dist-test-slave-ss4n"
server_key: "790e93910ef917ab97c99121a858c20f"
server_key_iv: "5c7dcdb1848c38d2238af6eedebbd7f0"
server_key_version: "encryptionkey@0"
I20260812 02:47:56.389222 6294 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:47:56.393348 6310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:56.395088 6294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:47:56.395337 6294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "ad1900d96efd4cb6953558c1c3dba0ca"
format_stamp: "Formatted at 2026-08-12 02:47:56 on dist-test-slave-ss4n"
server_key: "790e93910ef917ab97c99121a858c20f"
server_key_iv: "5c7dcdb1848c38d2238af6eedebbd7f0"
server_key_version: "encryptionkey@0"
I20260812 02:47:56.395599 6294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:56.412990 6294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:56.414077 6294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:56.414463 6294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:56.415704 6294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:56.417446 6294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:56.417613 6294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 02:47:56.417766 6294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:56.417862 6294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:56.456103 6294 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:34723
I20260812 02:47:56.456233 6446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:34723 every 8 connection(s)
I20260812 02:47:56.457640 6294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:47:56.466962 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 6294
I20260812 02:47:56.467170 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:47:56.467730 18195 external_mini_cluster.cc:1428] Setting key 5324b9bb24d33d81bde3bb0b8272e825
I20260812 02:47:56.471323 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:35075
--builtin_ntp_servers=127.17.196.212:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:47:56.472469 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:56.460572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:33327 (local address 127.17.196.254:35075)
0812 02:47:56.461094 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:56.461099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:56.462587 (+ 1488us) server_negotiation.cc:407] Connection header received
0812 02:47:56.464576 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:56.464582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:56.464694 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:56.464826 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:56.466062 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:56.466830 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:56.468715 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:56.469358 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:56.469653 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:56.471499 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:56.471506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:56.471510 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:47:56.471537 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:56.471594 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:47:56.471611 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:56.471737 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:56.471991 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 02:47:56.472103 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":258,"threads_started":1}
I20260812 02:47:56.473770 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:56.461304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:35075 (local address 127.17.196.194:33327)
0812 02:47:56.462401 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:56.462440 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:56.464218 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:56.465088 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:56.465097 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:56.465200 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:56.465879 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:56.465898 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:56.467014 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:56.467018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:56.468514 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:56.468534 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:56.469785 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:56.469796 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:56.470208 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:56.471055 (+ 847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:47:56.471058 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:56.471112 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:47:56.471265 (+ 153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:56.471736 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:56.471748 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:56.471760 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 02:47:56.471964 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 02:47:56.472430 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":367,"threads_started":1}
I20260812 02:47:56.475474 6447 heartbeater.cc:347] Connected to a master server at 127.17.196.254:35075
I20260812 02:47:56.475835 6447 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:56.476665 6447 heartbeater.cc:511] Master 127.17.196.254:35075 requested a full tablet report, sending...
I20260812 02:47:56.478614 6072 ts_manager.cc:194] Registered new tserver with Master: ad1900d96efd4cb6953558c1c3dba0ca (127.17.196.194:34723)
I20260812 02:47:56.479875 6072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.194:33327
W20260812 02:47:56.859709 6451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:47:56.860076 6451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:47:56.860239 6451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:47:56.868314 6451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:47:56.868481 6451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:47:56.868671 6451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:47:56.879077 6451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:35075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:47:56.880609 6451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:47:56.882773 6451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:47:56.896839 6464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:56.896881 6462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:47:56.896888 6460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:47:56.898221 6451 server_base.cc:1034] running on GCE node
I20260812 02:47:56.898942 6451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:47:56.899994 6451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:47:56.901314 6451 hybrid_clock.cc:648] HybridClock initialized: now 1786502876901259 us; error 46 us; skew 500 ppm
I20260812 02:47:56.901886 6451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:47:56.904775 6451 webserver.cc:460] Webserver started at http://127.17.196.195:43417/ using document root <none> and password file <none>
I20260812 02:47:56.905695 6451 fs_manager.cc:359] Metadata directory not provided
I20260812 02:47:56.905866 6451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:47:56.906244 6451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:47:56.909590 6451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74"
format_stamp: "Formatted at 2026-08-12 02:47:56 on dist-test-slave-ss4n"
server_key: "5ad1e7b88122467111b394a45cc516b2"
server_key_iv: "501418909233e9cea7d48f2d2ef2ef31"
server_key_version: "encryptionkey@0"
I20260812 02:47:56.910516 6451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74"
format_stamp: "Formatted at 2026-08-12 02:47:56 on dist-test-slave-ss4n"
server_key: "5ad1e7b88122467111b394a45cc516b2"
server_key_iv: "501418909233e9cea7d48f2d2ef2ef31"
server_key_version: "encryptionkey@0"
I20260812 02:47:56.916501 6451 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260812 02:47:56.920668 6470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:56.922400 6451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:47:56.922610 6451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74"
format_stamp: "Formatted at 2026-08-12 02:47:56 on dist-test-slave-ss4n"
server_key: "5ad1e7b88122467111b394a45cc516b2"
server_key_iv: "501418909233e9cea7d48f2d2ef2ef31"
server_key_version: "encryptionkey@0"
I20260812 02:47:56.922857 6451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:47:56.937026 6451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:47:56.938041 6451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:47:56.938374 6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:47:56.939467 6451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:47:56.941071 6451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:47:56.941208 6451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:56.941365 6451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:47:56.941457 6451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:47:56.968775 6290 heartbeater.cc:503] Master 127.17.196.254:35075 was elected leader, sending a full tablet report...
I20260812 02:47:56.979581 6451 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:39501
I20260812 02:47:56.979697 6606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:39501 every 8 connection(s)
I20260812 02:47:56.981016 6451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:47:56.988768 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 6451
I20260812 02:47:56.989023 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:47:56.989611 18195 external_mini_cluster.cc:1428] Setting key 70fbcd92ab086c5b3b99be8e76ef3c98
I20260812 02:47:56.994933 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:56.983975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:33139 (local address 127.17.196.254:35075)
0812 02:47:56.984457 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:56.984462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:56.985564 (+ 1102us) server_negotiation.cc:407] Connection header received
0812 02:47:56.987463 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:56.987469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:56.987588 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:56.987725 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:56.988936 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:56.989903 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:56.991723 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:56.992156 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:56.992298 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:56.993900 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:56.993905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:56.993908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:47:56.993927 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:56.993976 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:47:56.993989 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:56.994106 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:56.994357 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 02:47:56.994559 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":264,"threads_started":1}
I20260812 02:47:56.996042 6611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:56.984391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:35075 (local address 127.17.196.195:33139)
0812 02:47:56.985360 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:56.985400 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:56.987166 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:56.987972 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:56.987981 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:56.988082 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:47:56.988739 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:56.988758 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:56.990071 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:56.990076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:56.991556 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:56.991570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:56.992396 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:56.992404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:56.992657 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:56.993425 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:47:56.993428 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:47:56.993557 (+ 129us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:47:56.993669 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:47:56.994108 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:47:56.994122 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:47:56.994132 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 02:47:56.994350 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 02:47:56.994802 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":285,"threads_started":1}
I20260812 02:47:56.997593 6607 heartbeater.cc:347] Connected to a master server at 127.17.196.254:35075
I20260812 02:47:56.997939 6607 heartbeater.cc:464] Registering TS with master...
I20260812 02:47:56.998759 6607 heartbeater.cc:511] Master 127.17.196.254:35075 requested a full tablet report, sending...
I20260812 02:47:57.000351 6072 ts_manager.cc:194] Registered new tserver with Master: 1b7d93b6a4b64890a301d0d3fcb3cf74 (127.17.196.195:39501)
I20260812 02:47:57.001205 6072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.195:33139
I20260812 02:47:57.005365 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:47:57.016589 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.010126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.17.196.254:35075)
0812 02:47:57.010371 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.010380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.010664 (+ 284us) server_negotiation.cc:407] Connection header received
0812 02:47:57.010891 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.010904 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.010988 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.011110 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:57.011671 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.012428 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.013951 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.014395 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.014549 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.015800 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:57.015805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:57.015809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:47:57.015833 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:57.015870 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:47:57.015884 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:57.015991 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.016154 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.016260 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 02:47:57.021049 6072 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:47:57.028348 6072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:47:57.058100 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.042313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45262)
0812 02:47:57.043306 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.043390 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:57.045610 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:57.048433 (+ 2823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:57.048441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:57.048476 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:57.049021 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.049039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.051788 (+ 2749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.051795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.054952 (+ 3157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.054967 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.056867 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.056875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.057149 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.057217 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 02:47:57.057340 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 02:47:57.057606 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":1,"thread_start_us":355,"threads_started":1}
I20260812 02:47:57.058806 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.042950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:34723 (local address 127.0.0.1:41280)
0812 02:47:57.046656 (+ 3706us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.046685 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:57.046839 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:57.051116 (+ 4277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:57.051122 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:57.051167 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:57.051564 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.051577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.053754 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.053760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.055758 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.055774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.057881 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.057887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.058119 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.058158 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:47:57.058251 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 02:47:57.058399 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3506,"mutex_wait_us":28,"spinlock_wait_cycles":9600,"thread_start_us":3256,"threads_started":1}
I20260812 02:47:57.059387 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.042749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45262 (local address 127.17.196.193:38217)
0812 02:47:57.047576 (+ 4827us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.047586 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.047624 (+ 38us) server_negotiation.cc:407] Connection header received
0812 02:47:57.047768 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.047777 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.048134 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.049307 (+ 1173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:57.049693 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.051590 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.055206 (+ 3616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.056600 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.058556 (+ 1956us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.058742 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.058821 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.059060 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4557,"thread_start_us":4398,"threads_started":1}
I20260812 02:47:57.059727 6628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.043482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.17.196.194:34723)
0812 02:47:57.050397 (+ 6915us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.050406 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.050443 (+ 37us) server_negotiation.cc:407] Connection header received
0812 02:47:57.050579 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.050589 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.050878 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.051115 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:57.052239 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.053616 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.056005 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.057659 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.057838 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.058096 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.059209 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.059388 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6745,"thread_start_us":6605,"threads_started":1}
I20260812 02:47:57.060320 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.043534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:39501 (local address 127.0.0.1:36382)
0812 02:47:57.050981 (+ 7447us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.050999 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:57.051156 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:57.051878 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:57.051885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:57.051927 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:57.052499 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.052517 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.054758 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.054763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.056873 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.056887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.059553 (+ 2666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.059561 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.059758 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.059798 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:47:57.059896 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 02:47:57.060040 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7322,"thread_start_us":7189,"threads_started":1}
I20260812 02:47:57.061170 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.044202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36382 (local address 127.17.196.195:39501)
0812 02:47:57.050497 (+ 6295us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.050511 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.051299 (+ 788us) server_negotiation.cc:407] Connection header received
0812 02:47:57.051430 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.051441 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.051687 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.051893 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:57.052791 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.054578 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.057116 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.058995 (+ 1879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.059201 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.059411 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.060695 (+ 1284us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.060890 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5441,"thread_start_us":5261,"threads_started":1}
I20260812 02:47:57.064174 6372 tablet_service.cc:1467] Processing CreateTablet for tablet 510cf7065bba466282b4acb59df43aa0 (DEFAULT_TABLE table=test-table [id=421c1a72e344414d851ad39e37bd8c2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:57.064121 6212 tablet_service.cc:1467] Processing CreateTablet for tablet 510cf7065bba466282b4acb59df43aa0 (DEFAULT_TABLE table=test-table [id=421c1a72e344414d851ad39e37bd8c2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:57.064672 6527 tablet_service.cc:1467] Processing CreateTablet for tablet 510cf7065bba466282b4acb59df43aa0 (DEFAULT_TABLE table=test-table [id=421c1a72e344414d851ad39e37bd8c2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:47:57.066005 6212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510cf7065bba466282b4acb59df43aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:57.066116 6527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510cf7065bba466282b4acb59df43aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:57.066179 6372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510cf7065bba466282b4acb59df43aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:47:57.079322 6631 tablet_bootstrap.cc:492] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Bootstrap starting.
I20260812 02:47:57.082975 6632 tablet_bootstrap.cc:492] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Bootstrap starting.
I20260812 02:47:57.083254 6631 tablet_bootstrap.cc:654] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:57.085047 6631 log.cc:824] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:57.086969 6632 tablet_bootstrap.cc:654] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:57.088301 6632 log.cc:824] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:57.089226 6633 tablet_bootstrap.cc:492] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Bootstrap starting.
I20260812 02:47:57.089653 6631 tablet_bootstrap.cc:492] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: No bootstrap required, opened a new log
I20260812 02:47:57.089967 6631 ts_tablet_manager.cc:1397] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260812 02:47:57.091663 6632 tablet_bootstrap.cc:492] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: No bootstrap required, opened a new log
I20260812 02:47:57.091959 6632 ts_tablet_manager.cc:1397] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 02:47:57.093103 6633 tablet_bootstrap.cc:654] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Neither blocks nor log segments found. Creating new log.
I20260812 02:47:57.094458 6633 log.cc:824] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Log is configured to *not* fsync() on all Append() calls
I20260812 02:47:57.097944 6633 tablet_bootstrap.cc:492] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: No bootstrap required, opened a new log
I20260812 02:47:57.098264 6633 ts_tablet_manager.cc:1397] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 02:47:57.099948 6631 raft_consensus.cc:348] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.100397 6631 raft_consensus.cc:374] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:57.100503 6631 raft_consensus.cc:729] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7d93b6a4b64890a301d0d3fcb3cf74, State: Initialized, Role: FOLLOWER
I20260812 02:47:57.101208 6631 consensus_queue.cc:260] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.101588 6632 raft_consensus.cc:348] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.102041 6632 raft_consensus.cc:374] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:57.102165 6632 raft_consensus.cc:729] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d168bd1e95524674a072aead80b1100f, State: Initialized, Role: FOLLOWER
I20260812 02:47:57.102550 6607 heartbeater.cc:503] Master 127.17.196.254:35075 was elected leader, sending a full tablet report...
I20260812 02:47:57.102905 6632 consensus_queue.cc:260] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.103130 6631 ts_tablet_manager.cc:1428] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260812 02:47:57.104809 6632 ts_tablet_manager.cc:1428] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260812 02:47:57.109552 6633 raft_consensus.cc:348] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.110142 6633 raft_consensus.cc:374] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:47:57.110286 6633 raft_consensus.cc:729] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1900d96efd4cb6953558c1c3dba0ca, State: Initialized, Role: FOLLOWER
I20260812 02:47:57.110984 6633 consensus_queue.cc:260] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.112114 6447 heartbeater.cc:503] Master 127.17.196.254:35075 was elected leader, sending a full tablet report...
I20260812 02:47:57.112633 6633 ts_tablet_manager.cc:1428] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260812 02:47:57.191666 6291 tablet.cc:2369] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:47:57.210009 6448 tablet.cc:2369] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:47:57.218107 6637 raft_consensus.cc:482] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:47:57.218487 6637 raft_consensus.cc:504] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.220407 6637 leader_election.cc:288] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d168bd1e95524674a072aead80b1100f (127.17.196.193:38217), ad1900d96efd4cb6953558c1c3dba0ca (127.17.196.194:34723)
I20260812 02:47:57.227684 6628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.221163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:46191 (local address 127.17.196.194:34723)
0812 02:47:57.221391 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.221400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.221423 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:47:57.221540 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.221548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.221662 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.221767 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:57.222420 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.223356 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.225527 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.226456 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.226618 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.226678 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.227113 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.227351 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:47:57.227705 6611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.220687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:34723 (local address 127.17.196.195:46191)
0812 02:47:57.220944 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.220969 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:57.221121 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:57.221852 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:57.221861 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:57.221948 (+ 87us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:57.222218 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.222232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.223507 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.223513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.225318 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.225335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.226664 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.226670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.226890 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.226931 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 02:47:57.227062 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 02:47:57.227250 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":50,"spinlock_wait_cycles":16768}
I20260812 02:47:57.228343 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.220843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:33151 (local address 127.17.196.193:38217)
0812 02:47:57.221139 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.221150 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.221889 (+ 739us) server_negotiation.cc:407] Connection header received
0812 02:47:57.222132 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.222157 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.222260 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.222367 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:47:57.223019 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.223983 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.225930 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.227174 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.227333 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.227422 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.228030 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.228167 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:47:57.229027 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.220762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.17.196.195:33151)
0812 02:47:57.221766 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.221784 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:47:57.221949 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:47:57.222409 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:47:57.222414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:47:57.222440 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:47:57.222833 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.222846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.224189 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.224193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.225727 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:47:57.225744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.227526 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.227534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:47:57.227762 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.227801 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:47:57.228565 (+ 764us) client_negotiation.cc:240] Negotiation successful
0812 02:47:57.228723 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":2,"spinlock_wait_cycles":34816,"thread_start_us":610,"threads_started":1}
I20260812 02:47:57.228938 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510cf7065bba466282b4acb59df43aa0" candidate_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" is_pre_election: true
I20260812 02:47:57.229668 6394 raft_consensus.cc:2454] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7d93b6a4b64890a301d0d3fcb3cf74 in term 0.
I20260812 02:47:57.230556 6475 leader_election.cc:302] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7d93b6a4b64890a301d0d3fcb3cf74, ad1900d96efd4cb6953558c1c3dba0ca; no voters:
I20260812 02:47:57.230533 6236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510cf7065bba466282b4acb59df43aa0" candidate_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d168bd1e95524674a072aead80b1100f" is_pre_election: true
I20260812 02:47:57.231137 6637 raft_consensus.cc:2781] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:47:57.231329 6637 raft_consensus.cc:482] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:47:57.231226 6236 raft_consensus.cc:2454] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7d93b6a4b64890a301d0d3fcb3cf74 in term 0.
I20260812 02:47:57.231452 6637 raft_consensus.cc:3037] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 0 FOLLOWER]: Advancing to term 1
W20260812 02:47:57.233002 6608 tablet.cc:2369] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:47:57.234417 6637 raft_consensus.cc:504] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.235136 6637 leader_election.cc:288] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [CANDIDATE]: Term 1 election: Requested vote from peers d168bd1e95524674a072aead80b1100f (127.17.196.193:38217), ad1900d96efd4cb6953558c1c3dba0ca (127.17.196.194:34723)
I20260812 02:47:57.235805 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510cf7065bba466282b4acb59df43aa0" candidate_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad1900d96efd4cb6953558c1c3dba0ca"
I20260812 02:47:57.235859 6236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510cf7065bba466282b4acb59df43aa0" candidate_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d168bd1e95524674a072aead80b1100f"
I20260812 02:47:57.236071 6236 raft_consensus.cc:3037] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:57.236073 6394 raft_consensus.cc:3037] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:47:57.239864 6236 raft_consensus.cc:2454] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7d93b6a4b64890a301d0d3fcb3cf74 in term 1.
I20260812 02:47:57.239864 6394 raft_consensus.cc:2454] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7d93b6a4b64890a301d0d3fcb3cf74 in term 1.
I20260812 02:47:57.240468 6476 leader_election.cc:302] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b7d93b6a4b64890a301d0d3fcb3cf74, ad1900d96efd4cb6953558c1c3dba0ca, d168bd1e95524674a072aead80b1100f; no voters:
I20260812 02:47:57.240949 6637 raft_consensus.cc:2781] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:47:57.241430 6637 raft_consensus.cc:686] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 1 LEADER]: Becoming Leader. State: Replica: 1b7d93b6a4b64890a301d0d3fcb3cf74, State: Running, Role: LEADER
I20260812 02:47:57.242004 6637 consensus_queue.cc:237] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } }
I20260812 02:47:57.247403 6072 catalog_manager.cc:5495] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7d93b6a4b64890a301d0d3fcb3cf74 (127.17.196.195). New cstate: current_term: 1 leader_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7d93b6a4b64890a301d0d3fcb3cf74" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 39501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 } health_report { overall_health: UNKNOWN } } }
I20260812 02:47:57.300153 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.293851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.17.196.195:39501)
0812 02:47:57.294133 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.294138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.294159 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:47:57.294260 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.294264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.294358 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.294512 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:47:57.294999 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.295916 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.297438 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.298051 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.298170 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.298764 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:47:57.299423 (+ 659us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:47:57.299551 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.299741 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.299843 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260812 02:47:57.310333 6236 raft_consensus.cc:1261] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 1 FOLLOWER]: Refusing update from remote peer 1b7d93b6a4b64890a301d0d3fcb3cf74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:47:57.310323 6394 raft_consensus.cc:1261] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 1 FOLLOWER]: Refusing update from remote peer 1b7d93b6a4b64890a301d0d3fcb3cf74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:47:57.311565 6641 consensus_queue.cc:1035] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d168bd1e95524674a072aead80b1100f" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:47:57.311990 6637 consensus_queue.cc:1035] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad1900d96efd4cb6953558c1c3dba0ca" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:47:57.346364 6071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:47:57.346750 6071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:47:57.351970 6071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89b6d4da85374170ba7fdd874d9240ee: Sending DeleteTablet for 3 replicas of tablet 510cf7065bba466282b4acb59df43aa0
I20260812 02:47:57.353504 6212 tablet_service.cc:1514] Processing DeleteTablet for tablet 510cf7065bba466282b4acb59df43aa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:57 UTC) from {username='slave'} at 127.0.0.1:45262
I20260812 02:47:57.354667 6656 tablet_replica.cc:330] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: stopping tablet replica
I20260812 02:47:57.355175 6656 raft_consensus.cc:2229] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:47:57.355641 6656 raft_consensus.cc:2258] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:57.357437 6527 tablet_service.cc:1514] Processing DeleteTablet for tablet 510cf7065bba466282b4acb59df43aa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:57 UTC) from {username='slave'} at 127.0.0.1:36382
I20260812 02:47:57.357882 6372 tablet_service.cc:1514] Processing DeleteTablet for tablet 510cf7065bba466282b4acb59df43aa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:47:57 UTC) from {username='slave'} at 127.0.0.1:41280
I20260812 02:47:57.359129 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 6144
I20260812 02:47:57.366428 6656 ts_tablet_manager.cc:1905] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:47:57.373123 6658 tablet_replica.cc:330] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: stopping tablet replica
I20260812 02:47:57.373847 6658 raft_consensus.cc:2229] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:47:57.374661 6658 raft_consensus.cc:2258] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:57.377205 6658 ts_tablet_manager.cc:1905] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:47:57.378484 6657 tablet_replica.cc:330] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: stopping tablet replica
I20260812 02:47:57.379017 6657 raft_consensus.cc:2229] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:47:57.379751 6657 raft_consensus.cc:2258] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:47:57.382619 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.361313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.17.196.193:38217)
0812 02:47:57.361621 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.361632 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.371769 (+ 10137us) server_negotiation.cc:407] Connection header received
0812 02:47:57.371988 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.371998 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.372186 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.372364 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:57.373608 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.374894 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.377288 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.378071 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.378209 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.380139 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:57.380155 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:57.380176 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:47:57.381142 (+ 966us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:57.381223 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:47:57.381253 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:57.381370 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.381879 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.382066 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 02:47:57.384065 6656 ts_tablet_manager.cc:1918] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:57.384224 6284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:57.384313 6656 log.cc:1196] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/510cf7065bba466282b4acb59df43aa0
I20260812 02:47:57.385592 6657 ts_tablet_manager.cc:1905] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:47:57.394294 6658 ts_tablet_manager.cc:1918] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:57.394554 6658 log.cc:1196] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/510cf7065bba466282b4acb59df43aa0
I20260812 02:47:57.395251 6658 ts_tablet_manager.cc:1939] T 510cf7065bba466282b4acb59df43aa0 P ad1900d96efd4cb6953558c1c3dba0ca: Deleting consensus metadata
I20260812 02:47:57.396111 6657 ts_tablet_manager.cc:1918] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:47:57.396337 6657 log.cc:1196] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/510cf7065bba466282b4acb59df43aa0
I20260812 02:47:57.397198 6657 ts_tablet_manager.cc:1939] T 510cf7065bba466282b4acb59df43aa0 P 1b7d93b6a4b64890a301d0d3fcb3cf74: Deleting consensus metadata
I20260812 02:47:57.397750 6056 catalog_manager.cc:4841] TS ad1900d96efd4cb6953558c1c3dba0ca (127.17.196.194:34723): tablet 510cf7065bba466282b4acb59df43aa0 (table test-table [id=421c1a72e344414d851ad39e37bd8c2b]) successfully deleted
I20260812 02:47:57.399322 6057 catalog_manager.cc:4841] TS 1b7d93b6a4b64890a301d0d3fcb3cf74 (127.17.196.195:39501): tablet 510cf7065bba466282b4acb59df43aa0 (table test-table [id=421c1a72e344414d851ad39e37bd8c2b]) successfully deleted
I20260812 02:47:57.561390 6656 ts_tablet_manager.cc:1939] T 510cf7065bba466282b4acb59df43aa0 P d168bd1e95524674a072aead80b1100f: Deleting consensus metadata
I20260812 02:47:57.561450 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 6144
I20260812 02:47:57.587384 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 6294
W20260812 02:47:57.587486 6057 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.196.193:38217 (error 108)
I20260812 02:47:57.590090 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.589138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45290)
0812 02:47:57.589734 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.589865 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":307,"threads_started":1}
W20260812 02:47:57.590435 6057 catalog_manager.cc:4584] TS d168bd1e95524674a072aead80b1100f (127.17.196.193:38217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 510cf7065bba466282b4acb59df43aa0: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
I20260812 02:47:57.597693 6628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.588817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.17.196.194:34723)
0812 02:47:57.589056 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.589063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.589086 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:47:57.589161 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.589167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.589284 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.589405 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:57.590147 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.591271 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.593565 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.594176 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.594299 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.595857 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:57.595868 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:57.595885 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:47:57.596621 (+ 736us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:57.596683 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:47:57.596707 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:57.596830 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.597042 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.597169 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:47:57.599222 6440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:57.644354 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.643653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45306)
0812 02:47:57.643955 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.644101 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 02:47:57.645844 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.645311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45310)
0812 02:47:57.645572 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.645669 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 02:47:57.694657 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 6294
I20260812 02:47:57.706787 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.706243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45326)
0812 02:47:57.706460 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.706563 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 02:47:57.708014 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.707613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45336)
0812 02:47:57.707777 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.707865 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 02:47:57.725867 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 6451
I20260812 02:47:57.736665 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.726902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.17.196.195:39501)
0812 02:47:57.727111 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 02:47:57.727116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:47:57.727169 (+ 53us) server_negotiation.cc:407] Connection header received
0812 02:47:57.727429 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:47:57.727433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:47:57.727551 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:47:57.727722 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:47:57.728222 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.729701 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.731951 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:47:57.732769 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:47:57.732927 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:47:57.734770 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:47:57.734783 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:47:57.734805 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:47:57.735619 (+ 814us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:47:57.735703 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:47:57.735733 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:47:57.735857 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 02:47:57.736071 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 02:47:57.736204 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 02:47:57.737527 6597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:57.789001 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.788452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45340)
0812 02:47:57.788747 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.788836 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260812 02:47:57.790521 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:57.789767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45346)
0812 02:47:57.790015 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:57.790088 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260812 02:47:57.862955 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 6451
I20260812 02:47:57.887601 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 6038
I20260812 02:47:57.888823 6105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:47:58.011140 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:47:58.010498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:38217 (local address 127.0.0.1:45348)
0812 02:47:58.010770 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 02:47:58.010923 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 02:47:58.011336 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 6038
2026-08-12T02:47:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3277 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:47:58 dist-test-slave-ss4n krb5kdc[6669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:47:58 dist-test-slave-ss4n krb5kdc[6669](info): set up 2 sockets
Aug 12 02:47:58 dist-test-slave-ss4n krb5kdc[6669](info): commencing operation
krb5kdc: starting...
W20260812 02:47:59.288784 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.001s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:47:59 dist-test-slave-ss4n krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:47:59.647039 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39867
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:39867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:00.031713 6683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:00.032087 6683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:00.032292 6683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:00.040755 6683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:00.040902 6683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:00.040990 6683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:00.041083 6683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:00.041165 6683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:00.051952 6683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:39867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:39867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:00.053462 6683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:00.055028 6683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:00.069218 6688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:00.069260 6691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:00.070030 6689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:00.071338 6683 server_base.cc:1034] running on GCE node
I20260812 02:48:00.072449 6683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:00.074385 6683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:00.075698 6683 hybrid_clock.cc:648] HybridClock initialized: now 1786502880075631 us; error 46 us; skew 500 ppm
I20260812 02:48:00.076182 6683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:00 dist-test-slave-ss4n krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502880, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:00.080163 6683 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:48:00.082356 6683 webserver.cc:460] Webserver started at http://127.17.196.254:33577/ using document root <none> and password file <none>
I20260812 02:48:00.083163 6683 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:00.083325 6683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:00.083671 6683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:00.087128 6683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09"
format_stamp: "Formatted at 2026-08-12 02:48:00 on dist-test-slave-ss4n"
server_key: "1c30c60e4ca45c8e7a53facf72720393"
server_key_iv: "ce1d484c500f940e26fd9c7db78fccc9"
server_key_version: "encryptionkey@0"
I20260812 02:48:00.088049 6683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09"
format_stamp: "Formatted at 2026-08-12 02:48:00 on dist-test-slave-ss4n"
server_key: "1c30c60e4ca45c8e7a53facf72720393"
server_key_iv: "ce1d484c500f940e26fd9c7db78fccc9"
server_key_version: "encryptionkey@0"
I20260812 02:48:00.094933 6683 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:48:00.100306 6698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:00.102360 6683 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 02:48:00.102649 6683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09"
format_stamp: "Formatted at 2026-08-12 02:48:00 on dist-test-slave-ss4n"
server_key: "1c30c60e4ca45c8e7a53facf72720393"
server_key_iv: "ce1d484c500f940e26fd9c7db78fccc9"
server_key_version: "encryptionkey@0"
I20260812 02:48:00.102928 6683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:00.128086 6683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:00.133883 6683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:00.134232 6683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:00.159224 6683 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:39867
I20260812 02:48:00.159230 6758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:39867 every 8 connection(s)
I20260812 02:48:00.160878 6683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:00.163735 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 6683
I20260812 02:48:00.163930 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:00.164405 18195 external_mini_cluster.cc:1428] Setting key 361aec24668e76a45079d0e5585829b9
I20260812 02:48:00.165657 6759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:00.181972 6759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09: Bootstrap starting.
Aug 12 02:48:00 dist-test-slave-ss4n krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:00.185863 6759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:00.187170 6759 log.cc:824] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:00.190845 6759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09: No bootstrap required, opened a new log
I20260812 02:48:00.196446 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:00.166404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.17.196.254:39867)
0812 02:48:00.167393 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:00.167405 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:00.170417 (+ 3012us) server_negotiation.cc:407] Connection header received
0812 02:48:00.172570 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:00.172602 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:00.173155 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:00.173764 (+ 609us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:00.175799 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:00.177312 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:00.179050 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:00.179932 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:00.180202 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:00.186354 (+ 6152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:00.186366 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:00.186386 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:00.186440 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:00.190528 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:00.192120 (+ 1592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:00.192129 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:00.192139 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:00.192378 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:00.192891 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:00.192899 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:00.192901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:00.193678 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:00.194087 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:00.194628 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 02:48:00.195181 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":414,"threads_started":1}
I20260812 02:48:00.199554 6759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39867 } }
I20260812 02:48:00.200093 6759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:00.200230 6759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cca39bbbfbc4d2c9d2a9b5552abfb09, State: Initialized, Role: FOLLOWER
I20260812 02:48:00.200839 6759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39867 } }
I20260812 02:48:00.201143 6759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:00.201252 6759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:00.201392 6759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:00.204196 6759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39867 } }
I20260812 02:48:00.204862 6759 leader_election.cc:302] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cca39bbbfbc4d2c9d2a9b5552abfb09; no voters:
I20260812 02:48:00.205703 6759 leader_election.cc:288] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:00.205852 6764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:00.206638 6764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [term 1 LEADER]: Becoming Leader. State: Replica: 3cca39bbbfbc4d2c9d2a9b5552abfb09, State: Running, Role: LEADER
I20260812 02:48:00.207266 6764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39867 } }
I20260812 02:48:00.208218 6759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:00.211548 6766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cca39bbbfbc4d2c9d2a9b5552abfb09. Latest consensus state: current_term: 1 leader_uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39867 } } }
I20260812 02:48:00.212082 6766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:00.211752 6765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca39bbbfbc4d2c9d2a9b5552abfb09" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 39867 } } }
I20260812 02:48:00.212335 6765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:00.215184 6775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:00.224483 6775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:00.235891 6775 catalog_manager.cc:1347] Generated new cluster ID: b730b3344a8a44bf92f8fe69bfd64a79
I20260812 02:48:00.236088 6775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:00.248075 6775 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:00.249786 6775 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:00.264164 6775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09: Generated new TSK 0
I20260812 02:48:00.265415 6775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:48:00.374634 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39867
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:00.801836 6789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:00.802250 6789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:00.802429 6789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:00.811736 6789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:00.811890 6789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:00.812142 6789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:48:00.823721 6789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:00.825351 6789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:00.826994 6789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:00.843758 6798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:00.843637 6794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:00.843677 6795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:00.844666 6789 server_base.cc:1034] running on GCE node
I20260812 02:48:00.845456 6789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:00.846757 6789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:00.848043 6789 hybrid_clock.cc:648] HybridClock initialized: now 1786502880847977 us; error 48 us; skew 500 ppm
I20260812 02:48:00.848420 6789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:00 dist-test-slave-ss4n krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502880, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:00.852471 6789 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:48:00.854202 6789 webserver.cc:460] Webserver started at http://127.17.196.193:46435/ using document root <none> and password file <none>
I20260812 02:48:00.854964 6789 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:00.855125 6789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:00.855458 6789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:00.858999 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "57a2d07b586f4593af9572e1f2e3bbf6"
format_stamp: "Formatted at 2026-08-12 02:48:00 on dist-test-slave-ss4n"
server_key: "14e44bc1b951f3850cf7b747948b9ff2"
server_key_iv: "7f33ec2d5b4e6b8aa07466b3a1545c1c"
server_key_version: "encryptionkey@0"
I20260812 02:48:00.859903 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "57a2d07b586f4593af9572e1f2e3bbf6"
format_stamp: "Formatted at 2026-08-12 02:48:00 on dist-test-slave-ss4n"
server_key: "14e44bc1b951f3850cf7b747948b9ff2"
server_key_iv: "7f33ec2d5b4e6b8aa07466b3a1545c1c"
server_key_version: "encryptionkey@0"
I20260812 02:48:00.866189 6789 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:48:00.870476 6808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:00.872227 6789 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:48:00.872442 6789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "57a2d07b586f4593af9572e1f2e3bbf6"
format_stamp: "Formatted at 2026-08-12 02:48:00 on dist-test-slave-ss4n"
server_key: "14e44bc1b951f3850cf7b747948b9ff2"
server_key_iv: "7f33ec2d5b4e6b8aa07466b3a1545c1c"
server_key_version: "encryptionkey@0"
I20260812 02:48:00.872685 6789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:00.897917 6789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:00.903118 6789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:00.903484 6789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:00.904718 6789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:00.906345 6789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:00.906467 6789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:00.906617 6789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:00.906688 6789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:00.947433 6789 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:36375
I20260812 02:48:00.947510 6940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:36375 every 8 connection(s)
I20260812 02:48:00.949033 6789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:48:00.953969 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 6789
I20260812 02:48:00.954205 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:48:00.954721 18195 external_mini_cluster.cc:1428] Setting key 3ece61eb937bd9af26dd9d6dbea1b5d8
Aug 12 02:48:00 dist-test-slave-ss4n krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502880, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:00.972158 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:00.952329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:57871 (local address 127.17.196.254:39867)
0812 02:48:00.952999 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:00.953004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:00.953941 (+ 937us) server_negotiation.cc:407] Connection header received
0812 02:48:00.956270 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:00.956275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:00.956384 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:00.956515 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:00.958376 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:00.959131 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:00.960944 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:00.961374 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:00.961560 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:00.965408 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:00.965414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:00.965417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:00.965457 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:00.968474 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:00.969471 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:00.969476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:00.969519 (+ 43us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:00.969674 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:00.970167 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:00.970177 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:00.970179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:00.970477 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:00.970658 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:00.971483 (+ 825us) server_negotiation.cc:299] Negotiation successful
0812 02:48:00.971744 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":408,"threads_started":1}
I20260812 02:48:00.973274 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:00.952854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39867 (local address 127.17.196.193:57871)
0812 02:48:00.953761 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:00.953802 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:00.955969 (+ 2167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:00.956824 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:00.956834 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:00.957425 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:00.958152 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:00.958174 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:00.959318 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:00.959322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:00.960769 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:00.960786 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:00.961684 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:00.961693 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:00.961941 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:00.962714 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:48:00.962740 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:00.965133 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:00.968766 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:00.968777 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:00.968798 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:00.969313 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:00.969797 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:00.969803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:00.969807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:00.970036 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:00.970628 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:00.970638 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:00.971080 (+ 442us) client_negotiation.cc:769] Sending connection context
0812 02:48:00.971364 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 02:48:00.971997 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":328,"threads_started":1}
I20260812 02:48:00.975356 6942 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39867
I20260812 02:48:00.975813 6942 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:00.976835 6942 heartbeater.cc:511] Master 127.17.196.254:39867 requested a full tablet report, sending...
I20260812 02:48:00.980185 6717 ts_manager.cc:194] Registered new tserver with Master: 57a2d07b586f4593af9572e1f2e3bbf6 (127.17.196.193:36375)
I20260812 02:48:00.982930 6717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:57871
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:48:01.039371 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39867
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:01.463680 6949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:01.464035 6949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:01.464231 6949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:01.473258 6949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:01.473417 6949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:01.473724 6949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:48:01.485941 6949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:01.487525 6949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:01.489054 6949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:01.506108 6955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:01.506114 6956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:01.506619 6959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:01.507372 6949 server_base.cc:1034] running on GCE node
I20260812 02:48:01.508363 6949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:01.509732 6949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:01.511016 6949 hybrid_clock.cc:648] HybridClock initialized: now 1786502881510936 us; error 82 us; skew 500 ppm
I20260812 02:48:01.511379 6949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:01 dist-test-slave-ss4n krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502881, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:01.515803 6949 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:48:01.518003 6949 webserver.cc:460] Webserver started at http://127.17.196.194:35017/ using document root <none> and password file <none>
I20260812 02:48:01.518800 6949 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:01.518920 6949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:01.519240 6949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:01.523437 6949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "38467a63dd374db9ab42ed5466d6a589"
format_stamp: "Formatted at 2026-08-12 02:48:01 on dist-test-slave-ss4n"
server_key: "2456ff110a05c03bb9c9951fdeda578a"
server_key_iv: "103ab8fe6232d7b0f8170f5ec9904be5"
server_key_version: "encryptionkey@0"
I20260812 02:48:01.524720 6949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "38467a63dd374db9ab42ed5466d6a589"
format_stamp: "Formatted at 2026-08-12 02:48:01 on dist-test-slave-ss4n"
server_key: "2456ff110a05c03bb9c9951fdeda578a"
server_key_iv: "103ab8fe6232d7b0f8170f5ec9904be5"
server_key_version: "encryptionkey@0"
I20260812 02:48:01.532343 6949 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260812 02:48:01.538254 6967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:01.540266 6949 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260812 02:48:01.540458 6949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "38467a63dd374db9ab42ed5466d6a589"
format_stamp: "Formatted at 2026-08-12 02:48:01 on dist-test-slave-ss4n"
server_key: "2456ff110a05c03bb9c9951fdeda578a"
server_key_iv: "103ab8fe6232d7b0f8170f5ec9904be5"
server_key_version: "encryptionkey@0"
I20260812 02:48:01.540751 6949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:01.560984 6949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:01.567204 6949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:01.567551 6949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:01.568811 6949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:01.570824 6949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:01.570928 6949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:01.571038 6949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:01.571086 6949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:01.615720 6949 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:34299
I20260812 02:48:01.615814 7095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:34299 every 8 connection(s)
I20260812 02:48:01.617430 6949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:48:01.621234 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 6949
I20260812 02:48:01.621471 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:48:01.622210 18195 external_mini_cluster.cc:1428] Setting key 0e7cd53b202fea1193e3bf35f4f07da0
Aug 12 02:48:01 dist-test-slave-ss4n krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502881, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:01.643576 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:01.620718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:32839 (local address 127.17.196.254:39867)
0812 02:48:01.621364 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:01.621372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:01.622817 (+ 1445us) server_negotiation.cc:407] Connection header received
0812 02:48:01.625389 (+ 2572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:01.625395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:01.625554 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:01.625688 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:01.628008 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:01.628937 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:01.630980 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:01.631505 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:01.631685 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:01.636035 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:01.636050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:01.636053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:01.636094 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:01.639277 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:01.640269 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:01.640273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:01.640276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:01.640432 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:01.640971 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:01.640975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:01.640977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:01.641362 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:01.641574 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:01.642978 (+ 1404us) server_negotiation.cc:299] Negotiation successful
0812 02:48:01.643247 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":330,"threads_started":1}
I20260812 02:48:01.645311 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:01.621578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39867 (local address 127.17.196.194:32839)
0812 02:48:01.622603 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:01.622663 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:01.625063 (+ 2400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:01.626007 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:01.626018 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:01.626642 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:01.627354 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:01.627384 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:01.629177 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:01.629182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:01.630770 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:01.630788 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:01.631815 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:01.631824 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:01.632117 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:01.632980 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:48:01.633016 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:01.635712 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:01.639506 (+ 3794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:01.639519 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:01.639542 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:01.640088 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:01.640567 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:01.640573 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:01.640577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:01.640813 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:01.641600 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:01.641614 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:01.642275 (+ 661us) client_negotiation.cc:769] Sending connection context
0812 02:48:01.642640 (+ 365us) client_negotiation.cc:240] Negotiation successful
0812 02:48:01.643442 (+ 802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":408,"threads_started":1}
I20260812 02:48:01.647552 7096 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39867
I20260812 02:48:01.648015 7096 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:01.648931 7096 heartbeater.cc:511] Master 127.17.196.254:39867 requested a full tablet report, sending...
I20260812 02:48:01.650887 6717 ts_manager.cc:194] Registered new tserver with Master: 38467a63dd374db9ab42ed5466d6a589 (127.17.196.194:34299)
I20260812 02:48:01.652351 6717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:32839
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:48:01.707512 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39867
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:48:01.987586 6942 heartbeater.cc:503] Master 127.17.196.254:39867 was elected leader, sending a full tablet report...
W20260812 02:48:02.109968 7104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:02.110345 7104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:02.110513 7104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:02.118319 7104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:02.118502 7104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:02.118690 7104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:48:02.129272 7104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:39867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:02.130800 7104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:02.132294 7104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:02.147780 7109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:02.147712 7112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:02.148075 7110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:02.148713 7104 server_base.cc:1034] running on GCE node
I20260812 02:48:02.149427 7104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:02.150588 7104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:02.151842 7104 hybrid_clock.cc:648] HybridClock initialized: now 1786502882151793 us; error 47 us; skew 500 ppm
I20260812 02:48:02.152212 7104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:02 dist-test-slave-ss4n krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502882, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:02.162099 7104 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:48:02.164103 7104 webserver.cc:460] Webserver started at http://127.17.196.195:32989/ using document root <none> and password file <none>
I20260812 02:48:02.164885 7104 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:02.165043 7104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:02.165396 7104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:02.168658 7104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "44ae8954661043729605193d1d4a6c4d"
format_stamp: "Formatted at 2026-08-12 02:48:02 on dist-test-slave-ss4n"
server_key: "924a2f3b78eb222f016e4ba08dd42e7c"
server_key_iv: "617120029b34b5e08c058e3e59421143"
server_key_version: "encryptionkey@0"
I20260812 02:48:02.169538 7104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "44ae8954661043729605193d1d4a6c4d"
format_stamp: "Formatted at 2026-08-12 02:48:02 on dist-test-slave-ss4n"
server_key: "924a2f3b78eb222f016e4ba08dd42e7c"
server_key_iv: "617120029b34b5e08c058e3e59421143"
server_key_version: "encryptionkey@0"
I20260812 02:48:02.175364 7104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 02:48:02.179701 7119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:02.181229 7104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:48:02.181414 7104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "44ae8954661043729605193d1d4a6c4d"
format_stamp: "Formatted at 2026-08-12 02:48:02 on dist-test-slave-ss4n"
server_key: "924a2f3b78eb222f016e4ba08dd42e7c"
server_key_iv: "617120029b34b5e08c058e3e59421143"
server_key_version: "encryptionkey@0"
I20260812 02:48:02.181702 7104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:02.217722 7104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:02.222920 7104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:02.223284 7104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:02.224431 7104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:02.226224 7104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:02.226339 7104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:02.226503 7104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:02.226576 7104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:02.264798 7104 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:32883
I20260812 02:48:02.264868 7252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:32883 every 8 connection(s)
I20260812 02:48:02.266296 7104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:48:02.267395 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 7104
I20260812 02:48:02.267621 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:48:02.268287 18195 external_mini_cluster.cc:1428] Setting key b860051152c108052b44618aa7fe0456
Aug 12 02:48:02 dist-test-slave-ss4n krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502882, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:02.291997 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.269841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:54251 (local address 127.17.196.254:39867)
0812 02:48:02.270492 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.270500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.271895 (+ 1395us) server_negotiation.cc:407] Connection header received
0812 02:48:02.274129 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.274135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.274261 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.274399 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:02.276691 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.277636 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.279490 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.280068 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.280239 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.285210 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:02.285220 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:02.285225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:02.285271 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:02.288339 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:02.289255 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:02.289267 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:02.289269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:02.289405 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:02.289971 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:02.289977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:02.289984 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:02.290303 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:02.290472 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.291324 (+ 852us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.291582 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":363,"threads_started":1}
I20260812 02:48:02.293138 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.270687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:39867 (local address 127.17.196.195:54251)
0812 02:48:02.271716 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:02.271778 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:02.273813 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:02.274704 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:02.274714 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:02.275662 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:02.276379 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.276402 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.277822 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.277833 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.279294 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.279308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.280373 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.280382 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.280678 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.281630 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:48:02.281669 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:02.284942 (+ 3273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:02.288531 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:02.288542 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:02.288564 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:02.289096 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:02.289575 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:02.289581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:02.289585 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:02.289839 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:02.290460 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:02.290469 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:02.290886 (+ 417us) client_negotiation.cc:769] Sending connection context
0812 02:48:02.291254 (+ 368us) client_negotiation.cc:240] Negotiation successful
0812 02:48:02.291899 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":401,"threads_started":1}
I20260812 02:48:02.295221 7254 heartbeater.cc:347] Connected to a master server at 127.17.196.254:39867
I20260812 02:48:02.295646 7254 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:02.296550 7254 heartbeater.cc:511] Master 127.17.196.254:39867 requested a full tablet report, sending...
I20260812 02:48:02.298629 6716 ts_manager.cc:194] Registered new tserver with Master: 44ae8954661043729605193d1d4a6c4d (127.17.196.195:32883)
I20260812 02:48:02.299681 6716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:54251
I20260812 02:48:02.312110 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:48:02.335017 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.317855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.17.196.254:39867)
0812 02:48:02.318093 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.318098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.318732 (+ 634us) server_negotiation.cc:407] Connection header received
0812 02:48:02.318942 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.318947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.319024 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.319111 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:02.321087 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.322005 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.323719 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.324176 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.324306 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.327789 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:02.327797 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:02.327801 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:02.327852 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:02.331596 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:02.332769 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:02.332773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:02.332776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:02.332922 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:02.333405 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:02.333408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:02.333410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:02.333849 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:02.334067 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.334517 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.334755 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 02:48:02.340188 6716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:48:02.348356 6716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:48:02.388540 7272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.364945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:36375 (local address 127.0.0.1:34100)
0812 02:48:02.377918 (+ 12973us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:02.377939 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:02.378101 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:02.379224 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:02.379239 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:02.379300 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:02.379941 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.379959 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.382199 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.382206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.385057 (+ 2851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.385078 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.387321 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.387329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.387571 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.387639 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 02:48:02.387798 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 02:48:02.388034 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12765,"mutex_wait_us":52,"thread_start_us":10023,"threads_started":1}
I20260812 02:48:02.389748 7271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.365403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.17.196.193:36375)
0812 02:48:02.374892 (+ 9489us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.374908 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.378256 (+ 3348us) server_negotiation.cc:407] Connection header received
0812 02:48:02.378385 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.378394 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.378637 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.378878 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:02.380213 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.381981 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.385284 (+ 3303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.387033 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.387284 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.387501 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.389092 (+ 1591us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.389309 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9179,"thread_start_us":8916,"threads_started":1}
I20260812 02:48:02.390074 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.364789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:32883 (local address 127.0.0.1:36262)
0812 02:48:02.375544 (+ 10755us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:02.375629 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:02.375929 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:02.380950 (+ 5021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:02.380956 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:02.380987 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:02.381404 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.381420 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.383500 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.383506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.386159 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.386184 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.388806 (+ 2622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.388816 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.389009 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.389046 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:48:02.389158 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 02:48:02.389331 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10346,"mutex_wait_us":5,"thread_start_us":15103,"threads_started":1}
I20260812 02:48:02.390203 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.364947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.17.196.195:32883)
0812 02:48:02.377042 (+ 12095us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.377057 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.377838 (+ 781us) server_negotiation.cc:407] Connection header received
0812 02:48:02.377982 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.377994 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.378382 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.380584 (+ 2202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:02.381660 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.383289 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.386393 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.388096 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.388315 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.388501 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.389570 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.389789 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11753,"thread_start_us":11496,"threads_started":1}
I20260812 02:48:02.390851 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.377098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:34299 (local address 127.0.0.1:55206)
0812 02:48:02.380384 (+ 3286us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:02.380414 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:02.380609 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:02.381991 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:02.381996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:02.382031 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:02.382633 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.382647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.384871 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.384877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.387345 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.387364 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.389921 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.389930 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.390139 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.390182 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 02:48:02.390284 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 02:48:02.390426 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2496,"mutex_wait_us":138,"thread_start_us":426,"threads_started":1}
I20260812 02:48:02.391286 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.377395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.17.196.194:34299)
0812 02:48:02.381254 (+ 3859us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.381261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.381290 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:48:02.381383 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.381390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.381711 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.382000 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:02.382869 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.384697 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.387645 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.389311 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.389546 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.389752 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.390731 (+ 979us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.390944 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3677,"thread_start_us":3515,"threads_started":1}
I20260812 02:48:02.394791 7178 tablet_service.cc:1467] Processing CreateTablet for tablet 27c6a7fae5d1416ea65bb7439c71dba2 (DEFAULT_TABLE table=test-table [id=bdacc7cacc7b4daeba0144cf3d624b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:02.394582 6865 tablet_service.cc:1467] Processing CreateTablet for tablet 27c6a7fae5d1416ea65bb7439c71dba2 (DEFAULT_TABLE table=test-table [id=bdacc7cacc7b4daeba0144cf3d624b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:02.396139 7178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27c6a7fae5d1416ea65bb7439c71dba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:02.395546 7026 tablet_service.cc:1467] Processing CreateTablet for tablet 27c6a7fae5d1416ea65bb7439c71dba2 (DEFAULT_TABLE table=test-table [id=bdacc7cacc7b4daeba0144cf3d624b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:02.396353 6865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27c6a7fae5d1416ea65bb7439c71dba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:02.397113 7026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27c6a7fae5d1416ea65bb7439c71dba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:02.410629 7274 tablet_bootstrap.cc:492] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Bootstrap starting.
I20260812 02:48:02.414789 7279 tablet_bootstrap.cc:492] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Bootstrap starting.
I20260812 02:48:02.415216 7274 tablet_bootstrap.cc:654] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:02.417091 7278 tablet_bootstrap.cc:492] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Bootstrap starting.
I20260812 02:48:02.417883 7274 log.cc:824] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:02.419296 7279 tablet_bootstrap.cc:654] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:02.420655 7279 log.cc:824] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:02.421824 7278 tablet_bootstrap.cc:654] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:02.422127 7274 tablet_bootstrap.cc:492] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: No bootstrap required, opened a new log
I20260812 02:48:02.422484 7274 ts_tablet_manager.cc:1397] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260812 02:48:02.423085 7278 log.cc:824] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:02.424362 7279 tablet_bootstrap.cc:492] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: No bootstrap required, opened a new log
I20260812 02:48:02.424677 7279 ts_tablet_manager.cc:1397] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260812 02:48:02.425976 7278 tablet_bootstrap.cc:492] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: No bootstrap required, opened a new log
I20260812 02:48:02.426259 7278 ts_tablet_manager.cc:1397] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 02:48:02.434197 7278 raft_consensus.cc:348] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.434172 7274 raft_consensus.cc:348] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.434644 7278 raft_consensus.cc:374] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:02.434679 7274 raft_consensus.cc:374] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:02.434753 7278 raft_consensus.cc:729] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57a2d07b586f4593af9572e1f2e3bbf6, State: Initialized, Role: FOLLOWER
I20260812 02:48:02.434777 7274 raft_consensus.cc:729] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ae8954661043729605193d1d4a6c4d, State: Initialized, Role: FOLLOWER
I20260812 02:48:02.435451 7278 consensus_queue.cc:260] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.435560 7274 consensus_queue.cc:260] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.436012 7279 raft_consensus.cc:348] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.436636 7279 raft_consensus.cc:374] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:02.436756 7279 raft_consensus.cc:729] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38467a63dd374db9ab42ed5466d6a589, State: Initialized, Role: FOLLOWER
I20260812 02:48:02.437456 7278 ts_tablet_manager.cc:1428] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260812 02:48:02.437608 7279 consensus_queue.cc:260] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.438562 7254 heartbeater.cc:503] Master 127.17.196.254:39867 was elected leader, sending a full tablet report...
I20260812 02:48:02.440459 7274 ts_tablet_manager.cc:1428] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260812 02:48:02.441651 7096 heartbeater.cc:503] Master 127.17.196.254:39867 was elected leader, sending a full tablet report...
I20260812 02:48:02.442456 7279 ts_tablet_manager.cc:1428] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Time spent starting tablet: real 0.018s user 0.016s sys 0.004s
W20260812 02:48:02.452611 6944 tablet.cc:2369] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:48:02.518473 7255 tablet.cc:2369] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:48:02.620344 7098 tablet.cc:2369] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:48:02.670792 7285 raft_consensus.cc:482] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:48:02.671159 7285 raft_consensus.cc:504] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.673126 7285 leader_election.cc:288] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44ae8954661043729605193d1d4a6c4d (127.17.196.195:32883), 57a2d07b586f4593af9572e1f2e3bbf6 (127.17.196.193:36375)
I20260812 02:48:02.683503 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.673882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:48639 (local address 127.17.196.195:32883)
0812 02:48:02.674095 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.674102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.675080 (+ 978us) server_negotiation.cc:407] Connection header received
0812 02:48:02.675357 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.675363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.675456 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.675594 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:02.676650 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.678015 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.681275 (+ 3260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.682282 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.682539 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.682640 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.683005 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.683181 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:48:02.683550 7271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.673778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:55063 (local address 127.17.196.193:36375)
0812 02:48:02.674051 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.674059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.674981 (+ 922us) server_negotiation.cc:407] Connection header received
0812 02:48:02.675353 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.675361 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.675456 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.675584 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:02.676650 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.678035 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.681404 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.682421 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.682656 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.682752 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.683092 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.683255 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 02:48:02.683772 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.673673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:36375 (local address 127.17.196.194:55063)
0812 02:48:02.674826 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:02.674852 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:02.675086 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:02.675672 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:02.675681 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:02.675720 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:02.676430 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.676444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.678284 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.678289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.681170 (+ 2881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.681190 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.682713 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.682721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.682902 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.682929 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 02:48:02.683053 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 02:48:02.683249 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":311,"thread_start_us":632,"threads_started":1}
I20260812 02:48:02.683820 7286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.673689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:32883 (local address 127.17.196.194:48639)
0812 02:48:02.674955 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:02.674973 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:02.675160 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:02.675728 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:02.675732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:02.675752 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:02.676340 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.676363 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.678303 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.678309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.681018 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:02.681046 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.682598 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.682608 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:02.682803 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.682833 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 02:48:02.682969 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 02:48:02.683176 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1121,"mutex_wait_us":60,"spinlock_wait_cycles":30592,"thread_start_us":589,"threads_started":1}
I20260812 02:48:02.685344 7199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27c6a7fae5d1416ea65bb7439c71dba2" candidate_uuid: "38467a63dd374db9ab42ed5466d6a589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae8954661043729605193d1d4a6c4d" is_pre_election: true
I20260812 02:48:02.685344 6888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27c6a7fae5d1416ea65bb7439c71dba2" candidate_uuid: "38467a63dd374db9ab42ed5466d6a589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" is_pre_election: true
I20260812 02:48:02.686007 7199 raft_consensus.cc:2454] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38467a63dd374db9ab42ed5466d6a589 in term 0.
I20260812 02:48:02.686007 6888 raft_consensus.cc:2454] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38467a63dd374db9ab42ed5466d6a589 in term 0.
I20260812 02:48:02.686980 6971 leader_election.cc:302] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 38467a63dd374db9ab42ed5466d6a589, 44ae8954661043729605193d1d4a6c4d, 57a2d07b586f4593af9572e1f2e3bbf6; no voters:
I20260812 02:48:02.687661 7285 raft_consensus.cc:2781] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:48:02.687868 7285 raft_consensus.cc:482] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:48:02.688058 7285 raft_consensus.cc:3037] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:02.690840 7285 raft_consensus.cc:504] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.691649 7285 leader_election.cc:288] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [CANDIDATE]: Term 1 election: Requested vote from peers 44ae8954661043729605193d1d4a6c4d (127.17.196.195:32883), 57a2d07b586f4593af9572e1f2e3bbf6 (127.17.196.193:36375)
I20260812 02:48:02.692478 6888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27c6a7fae5d1416ea65bb7439c71dba2" candidate_uuid: "38467a63dd374db9ab42ed5466d6a589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57a2d07b586f4593af9572e1f2e3bbf6"
I20260812 02:48:02.692672 7199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27c6a7fae5d1416ea65bb7439c71dba2" candidate_uuid: "38467a63dd374db9ab42ed5466d6a589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae8954661043729605193d1d4a6c4d"
I20260812 02:48:02.692745 6888 raft_consensus.cc:3037] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:02.692926 7199 raft_consensus.cc:3037] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:02.695955 7199 raft_consensus.cc:2454] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38467a63dd374db9ab42ed5466d6a589 in term 1.
I20260812 02:48:02.696025 6888 raft_consensus.cc:2454] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38467a63dd374db9ab42ed5466d6a589 in term 1.
I20260812 02:48:02.696727 6971 leader_election.cc:302] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 38467a63dd374db9ab42ed5466d6a589, 44ae8954661043729605193d1d4a6c4d, 57a2d07b586f4593af9572e1f2e3bbf6; no voters:
I20260812 02:48:02.697098 7285 raft_consensus.cc:2781] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:02.697978 7285 raft_consensus.cc:686] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 1 LEADER]: Becoming Leader. State: Replica: 38467a63dd374db9ab42ed5466d6a589, State: Running, Role: LEADER
I20260812 02:48:02.698653 7285 consensus_queue.cc:237] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } }
I20260812 02:48:02.704487 6714 catalog_manager.cc:5495] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38467a63dd374db9ab42ed5466d6a589 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "38467a63dd374db9ab42ed5466d6a589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38467a63dd374db9ab42ed5466d6a589" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 34299 } health_report { overall_health: HEALTHY } } }
I20260812 02:48:02.766999 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.760613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.17.196.194:34299)
0812 02:48:02.760882 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.760890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.761009 (+ 119us) server_negotiation.cc:407] Connection header received
0812 02:48:02.761295 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.761299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.761397 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.761641 (+ 244us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:48:02.762105 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.762836 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.764405 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.764855 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.765004 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.765745 (+ 741us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:48:02.766291 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:48:02.766489 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.766675 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.766726 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 02:48:02.777356 7199 raft_consensus.cc:1261] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 1 FOLLOWER]: Refusing update from remote peer 38467a63dd374db9ab42ed5466d6a589: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:48:02.777382 6888 raft_consensus.cc:1261] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 1 FOLLOWER]: Refusing update from remote peer 38467a63dd374db9ab42ed5466d6a589: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:48:02.778990 7288 consensus_queue.cc:1035] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57a2d07b586f4593af9572e1f2e3bbf6" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:48:02.779446 7285 consensus_queue.cc:1035] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ae8954661043729605193d1d4a6c4d" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:48:02.796690 7293 mvcc.cc:205] Tried to move back new op lower bound from 7317515807842295808 to 7317515807538438144. Current Snapshot: MvccSnapshot[applied={T|T < 7317515807842295808}]
I20260812 02:48:02.814574 6714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:48:02.814953 6714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:48:02.820042 6714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cca39bbbfbc4d2c9d2a9b5552abfb09: Sending DeleteTablet for 3 replicas of tablet 27c6a7fae5d1416ea65bb7439c71dba2
I20260812 02:48:02.821475 7178 tablet_service.cc:1514] Processing DeleteTablet for tablet 27c6a7fae5d1416ea65bb7439c71dba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:02 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:36262
I20260812 02:48:02.821820 7026 tablet_service.cc:1514] Processing DeleteTablet for tablet 27c6a7fae5d1416ea65bb7439c71dba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:02 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:55206
I20260812 02:48:02.821951 6865 tablet_service.cc:1514] Processing DeleteTablet for tablet 27c6a7fae5d1416ea65bb7439c71dba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:02 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:34100
I20260812 02:48:02.824689 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 6789
I20260812 02:48:02.832813 7306 tablet_replica.cc:330] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: stopping tablet replica
I20260812 02:48:02.833628 7306 raft_consensus.cc:2229] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:48:02.834388 7306 raft_consensus.cc:2258] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:02.835415 7304 tablet_replica.cc:330] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: stopping tablet replica
I20260812 02:48:02.836086 7304 raft_consensus.cc:2229] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:48:02.837772 7305 tablet_replica.cc:330] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: stopping tablet replica
I20260812 02:48:02.838163 7306 ts_tablet_manager.cc:1905] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:02.838851 7305 raft_consensus.cc:2229] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:48:02.839838 7305 raft_consensus.cc:2258] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:02.841538 7304 raft_consensus.cc:2258] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:02.844152 7305 ts_tablet_manager.cc:1905] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:02.847967 7306 ts_tablet_manager.cc:1918] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:02.848287 7306 log.cc:1196] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/27c6a7fae5d1416ea65bb7439c71dba2
I20260812 02:48:02.849042 7306 ts_tablet_manager.cc:1939] T 27c6a7fae5d1416ea65bb7439c71dba2 P 57a2d07b586f4593af9572e1f2e3bbf6: Deleting consensus metadata
Aug 12 02:48:02 dist-test-slave-ss4n krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:48:02.851600 7304 ts_tablet_manager.cc:1905] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:02.852003 6700 catalog_manager.cc:4841] TS 57a2d07b586f4593af9572e1f2e3bbf6 (127.17.196.193:36375): tablet 27c6a7fae5d1416ea65bb7439c71dba2 (table test-table [id=bdacc7cacc7b4daeba0144cf3d624b23]) successfully deleted
I20260812 02:48:02.856973 7305 ts_tablet_manager.cc:1918] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:02.857424 7305 log.cc:1196] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/27c6a7fae5d1416ea65bb7439c71dba2
I20260812 02:48:02.858466 7305 ts_tablet_manager.cc:1939] T 27c6a7fae5d1416ea65bb7439c71dba2 P 38467a63dd374db9ab42ed5466d6a589: Deleting consensus metadata
I20260812 02:48:02.860622 6700 catalog_manager.cc:4841] TS 38467a63dd374db9ab42ed5466d6a589 (127.17.196.194:34299): tablet 27c6a7fae5d1416ea65bb7439c71dba2 (table test-table [id=bdacc7cacc7b4daeba0144cf3d624b23]) successfully deleted
I20260812 02:48:02.861276 7304 ts_tablet_manager.cc:1918] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:02.861526 7304 log.cc:1196] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/27c6a7fae5d1416ea65bb7439c71dba2
I20260812 02:48:02.862054 7304 ts_tablet_manager.cc:1939] T 27c6a7fae5d1416ea65bb7439c71dba2 P 44ae8954661043729605193d1d4a6c4d: Deleting consensus metadata
I20260812 02:48:02.862490 7271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.825942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.17.196.193:36375)
0812 02:48:02.826234 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.826246 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.836783 (+ 10537us) server_negotiation.cc:407] Connection header received
0812 02:48:02.837091 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.837099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.837215 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.837345 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:02.840312 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.841836 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.844324 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.845200 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:02.845337 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:02.852281 (+ 6944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:02.852300 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:02.852322 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:02.852383 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:02.857263 (+ 4880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:02.858894 (+ 1631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:02.858907 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:02.858910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:02.859178 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:02.859854 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:02.859860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:02.859864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:02.860470 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:02.860878 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:02.861680 (+ 802us) server_negotiation.cc:299] Negotiation successful
0812 02:48:02.861986 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 02:48:02.863933 6702 catalog_manager.cc:4841] TS 44ae8954661043729605193d1d4a6c4d (127.17.196.195:32883): tablet 27c6a7fae5d1416ea65bb7439c71dba2 (table test-table [id=bdacc7cacc7b4daeba0144cf3d624b23]) successfully deleted
I20260812 02:48:02.864408 6935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:02.970405 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 6789
I20260812 02:48:02.994939 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 6949
Aug 12 02:48:03 dist-test-slave-ss4n krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:48:03.016414 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:02.995902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.17.196.194:34299)
0812 02:48:02.996135 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:02.996143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:02.996167 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:48:02.996274 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:02.996280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:02.996369 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:02.996499 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:02.998252 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:02.999450 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:03.001038 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:03.002100 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:03.002274 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:03.007795 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:03.007843 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:03.007868 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:03.007925 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:03.012534 (+ 4609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:03.013902 (+ 1368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:03.013912 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:03.013933 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:03.014173 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:03.014647 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:03.014652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:03.014654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:03.015033 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:03.015449 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:03.015751 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 02:48:03.016033 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 02:48:03.017151 7091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:03.139562 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 6949
I20260812 02:48:03.165053 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 7104
Aug 12 02:48:03 dist-test-slave-ss4n krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:48:03.185745 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:03.165919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.17.196.195:32883)
0812 02:48:03.166163 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:03.166170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:03.166196 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:48:03.166378 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:03.166384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:03.166474 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:03.166595 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:03.168353 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:03.169652 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:03.171100 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:03.171815 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:03.171962 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:03.178165 (+ 6203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:03.178181 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:03.178197 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:03.178250 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:03.182047 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:03.183124 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:03.183134 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:03.183137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:03.183332 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:03.183870 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:03.183874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:03.183877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:03.184326 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:03.184592 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:03.185058 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 02:48:03.185259 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260812 02:48:03.186590 7247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:03.282274 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 7104
I20260812 02:48:03.305575 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 6683
I20260812 02:48:03.306919 6754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:03.427129 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 6683
2026-08-12T02:48:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5417 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T02:48:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:03Z Disabled control of system clock
I20260812 02:48:03.533226 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37827
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:37827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:48:03.935122 7319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:03.935490 7319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:03.935623 7319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:03.944877 7319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:03.945034 7319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:03.945079 7319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:03.945200 7319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:03.945276 7319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:03.956574 7319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:37827
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:03.958272 7319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:03.960572 7319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:03.974710 7326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:03.974848 7327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:03.974833 7329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:03.980033 7319 server_base.cc:1034] running on GCE node
I20260812 02:48:03.981112 7319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:03.983139 7319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:03.984412 7319 hybrid_clock.cc:648] HybridClock initialized: now 1786502883984341 us; error 54 us; skew 500 ppm
I20260812 02:48:03.984795 7319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:48:03.987632 7319 webserver.cc:460] Webserver started at http://127.17.196.254:33623/ using document root <none> and password file <none>
I20260812 02:48:03.988445 7319 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:03.988550 7319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:03.988857 7319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:03.992411 7319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "113ba9b6e2214a9d92dfa97a32f21fd4"
format_stamp: "Formatted at 2026-08-12 02:48:03 on dist-test-slave-ss4n"
server_key: "49653d9e4851e8be6f6373a10305e981"
server_key_iv: "8a957f486f0fe610e241a37c9fc6d07d"
server_key_version: "encryptionkey@0"
I20260812 02:48:03.993396 7319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "113ba9b6e2214a9d92dfa97a32f21fd4"
format_stamp: "Formatted at 2026-08-12 02:48:03 on dist-test-slave-ss4n"
server_key: "49653d9e4851e8be6f6373a10305e981"
server_key_iv: "8a957f486f0fe610e241a37c9fc6d07d"
server_key_version: "encryptionkey@0"
I20260812 02:48:03.999966 7319 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 02:48:04.004604 7340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:04.006448 7319 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 02:48:04.006706 7319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "113ba9b6e2214a9d92dfa97a32f21fd4"
format_stamp: "Formatted at 2026-08-12 02:48:03 on dist-test-slave-ss4n"
server_key: "49653d9e4851e8be6f6373a10305e981"
server_key_iv: "8a957f486f0fe610e241a37c9fc6d07d"
server_key_version: "encryptionkey@0"
I20260812 02:48:04.007017 7319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:04.032121 7319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:04.033192 7319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:04.033588 7319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:04.057281 7319 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:37827
I20260812 02:48:04.057304 7400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:37827 every 8 connection(s)
I20260812 02:48:04.059005 7319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:04.063704 7401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:04.065151 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 7319
I20260812 02:48:04.065411 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:04.066004 18195 external_mini_cluster.cc:1428] Setting key 634f17b4627bc2944549598b292fc3ab
I20260812 02:48:04.079871 7401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4: Bootstrap starting.
I20260812 02:48:04.082794 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:04.068380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.17.196.254:37827)
0812 02:48:04.070024 (+ 1644us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:04.070039 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:04.070103 (+ 64us) server_negotiation.cc:407] Connection header received
0812 02:48:04.072023 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:04.072054 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:04.072687 (+ 633us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:04.073209 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:04.073962 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:04.075414 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:04.077044 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:04.078040 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:04.078286 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:04.079842 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:04.079855 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:04.079874 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:04.080579 (+ 705us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:04.080735 (+ 156us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:04.080764 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:04.080917 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:04.081209 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 02:48:04.081674 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":971,"thread_start_us":701,"threads_started":1}
I20260812 02:48:04.083887 7401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:04.085246 7401 log.cc:824] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:04.088665 7401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4: No bootstrap required, opened a new log
I20260812 02:48:04.097317 7401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113ba9b6e2214a9d92dfa97a32f21fd4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37827 } }
I20260812 02:48:04.098027 7401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:04.098193 7401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 113ba9b6e2214a9d92dfa97a32f21fd4, State: Initialized, Role: FOLLOWER
I20260812 02:48:04.099170 7401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113ba9b6e2214a9d92dfa97a32f21fd4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37827 } }
I20260812 02:48:04.099534 7401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:04.099720 7401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:04.099922 7401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:04.102699 7401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113ba9b6e2214a9d92dfa97a32f21fd4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37827 } }
I20260812 02:48:04.103358 7401 leader_election.cc:302] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 113ba9b6e2214a9d92dfa97a32f21fd4; no voters:
I20260812 02:48:04.104148 7401 leader_election.cc:288] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:04.104417 7407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:04.105285 7407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 113ba9b6e2214a9d92dfa97a32f21fd4, State: Running, Role: LEADER
I20260812 02:48:04.106038 7407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113ba9b6e2214a9d92dfa97a32f21fd4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37827 } }
I20260812 02:48:04.107091 7401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:04.109901 7408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "113ba9b6e2214a9d92dfa97a32f21fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113ba9b6e2214a9d92dfa97a32f21fd4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37827 } } }
I20260812 02:48:04.110674 7408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:04.112766 7409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 113ba9b6e2214a9d92dfa97a32f21fd4. Latest consensus state: current_term: 1 leader_uuid: "113ba9b6e2214a9d92dfa97a32f21fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113ba9b6e2214a9d92dfa97a32f21fd4" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37827 } } }
I20260812 02:48:04.113268 7409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:04.114787 7417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:04.120172 7417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:04.131275 7417 catalog_manager.cc:1347] Generated new cluster ID: 3f0ca2abf6ed43a7b1f2d98c074e5247
I20260812 02:48:04.131429 7417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:04.162514 7417 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:04.164836 7417 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:04.179307 7417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4: Generated new TSK 0
I20260812 02:48:04.180611 7417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:04.186378 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37827
--builtin_ntp_servers=127.17.196.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:48:04.586014 7432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:04.586345 7432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:04.586437 7432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:04.594254 7432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:04.594380 7432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:04.594496 7432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:48:04.604462 7432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:04.605894 7432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:04.607928 7432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:04.623493 7438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:04.623560 7437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:04.623512 7440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:04.625078 7432 server_base.cc:1034] running on GCE node
I20260812 02:48:04.625965 7432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:04.627192 7432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:04.628491 7432 hybrid_clock.cc:648] HybridClock initialized: now 1786502884628429 us; error 50 us; skew 500 ppm
I20260812 02:48:04.628943 7432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:48:04.631767 7432 webserver.cc:460] Webserver started at http://127.17.196.193:42805/ using document root <none> and password file <none>
I20260812 02:48:04.632567 7432 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:04.632720 7432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:04.633131 7432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:04.636523 7432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "29acd0821ea54465bead0c3aa23e7a5a"
format_stamp: "Formatted at 2026-08-12 02:48:04 on dist-test-slave-ss4n"
server_key: "13ef905e8784bec731a33d93dda71376"
server_key_iv: "03e6e3bf4d6bf3408f2e4f4705395b36"
server_key_version: "encryptionkey@0"
I20260812 02:48:04.637622 7432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "29acd0821ea54465bead0c3aa23e7a5a"
format_stamp: "Formatted at 2026-08-12 02:48:04 on dist-test-slave-ss4n"
server_key: "13ef905e8784bec731a33d93dda71376"
server_key_iv: "03e6e3bf4d6bf3408f2e4f4705395b36"
server_key_version: "encryptionkey@0"
I20260812 02:48:04.643543 7432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:48:04.647434 7446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:04.648932 7432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:48:04.649199 7432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "29acd0821ea54465bead0c3aa23e7a5a"
format_stamp: "Formatted at 2026-08-12 02:48:04 on dist-test-slave-ss4n"
server_key: "13ef905e8784bec731a33d93dda71376"
server_key_iv: "03e6e3bf4d6bf3408f2e4f4705395b36"
server_key_version: "encryptionkey@0"
I20260812 02:48:04.649523 7432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:04.673544 7432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:04.674647 7432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:04.675033 7432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:04.676229 7432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:04.678086 7432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:04.678221 7432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:04.678367 7432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:04.678469 7432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:04.719501 7432 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:44481
I20260812 02:48:04.719610 7581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:44481 every 8 connection(s)
I20260812 02:48:04.721079 7432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:48:04.725419 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 7432
I20260812 02:48:04.725675 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:48:04.726329 18195 external_mini_cluster.cc:1428] Setting key 39c5ba74adae94ed1b8917b9f78d395c
I20260812 02:48:04.730022 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37827
--builtin_ntp_servers=127.17.196.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:48:04.738584 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:04.724149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:58217 (local address 127.17.196.254:37827)
0812 02:48:04.724681 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:04.724688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:04.726376 (+ 1688us) server_negotiation.cc:407] Connection header received
0812 02:48:04.728837 (+ 2461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:04.728842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:04.728930 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:04.729048 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:04.730955 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:04.731776 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:04.734813 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:04.735292 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:04.735409 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:04.737553 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:04.737566 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:04.737570 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:04.737597 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:04.737650 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:04.737667 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:04.737754 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:04.738090 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 02:48:04.738195 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":239,"threads_started":1}
I20260812 02:48:04.740300 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:04.724795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37827 (local address 127.17.196.193:58217)
0812 02:48:04.726137 (+ 1342us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:04.726198 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:04.728407 (+ 2209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:04.729426 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:04.729441 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:04.729647 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:04.730714 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:04.730751 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:04.731981 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:04.731991 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:04.734595 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:04.734622 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:04.735593 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:04.735605 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:04.736003 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:04.736955 (+ 952us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:48:04.736961 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:04.737032 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:48:04.737176 (+ 144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:04.737781 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:04.737797 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:04.737811 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 02:48:04.738040 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 02:48:04.738611 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":346,"threads_started":1}
I20260812 02:48:04.742194 7582 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37827
I20260812 02:48:04.742551 7582 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:04.743386 7582 heartbeater.cc:511] Master 127.17.196.254:37827 requested a full tablet report, sending...
I20260812 02:48:04.746313 7361 ts_manager.cc:194] Registered new tserver with Master: 29acd0821ea54465bead0c3aa23e7a5a (127.17.196.193:44481)
I20260812 02:48:04.748019 7361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.193:58217
W20260812 02:48:05.116323 7586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:05.116680 7586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:05.116820 7586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:05.124780 7586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:05.124922 7586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:05.125109 7586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:48:05.135298 7586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:05.136832 7586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:05.139101 7586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:05.156198 7596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:05.156309 7593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:05.156318 7594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:05.158012 7586 server_base.cc:1034] running on GCE node
I20260812 02:48:05.159085 7586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:05.160780 7586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:05.162081 7586 hybrid_clock.cc:648] HybridClock initialized: now 1786502885162038 us; error 80 us; skew 500 ppm
I20260812 02:48:05.162520 7586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:48:05.165566 7586 webserver.cc:460] Webserver started at http://127.17.196.194:45849/ using document root <none> and password file <none>
I20260812 02:48:05.166491 7586 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:05.166649 7586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:05.167037 7586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:05.170763 7586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "58e326985c734aad8b71dcd03dcb4e46"
format_stamp: "Formatted at 2026-08-12 02:48:05 on dist-test-slave-ss4n"
server_key: "61cdb2fd1260d351cef0fe61ed5dd1ac"
server_key_iv: "be7ae9b998e43e6b238937d0971b3f4a"
server_key_version: "encryptionkey@0"
I20260812 02:48:05.172027 7586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "58e326985c734aad8b71dcd03dcb4e46"
format_stamp: "Formatted at 2026-08-12 02:48:05 on dist-test-slave-ss4n"
server_key: "61cdb2fd1260d351cef0fe61ed5dd1ac"
server_key_iv: "be7ae9b998e43e6b238937d0971b3f4a"
server_key_version: "encryptionkey@0"
I20260812 02:48:05.178534 7586 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:48:05.183188 7602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:05.184875 7586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260812 02:48:05.185110 7586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "58e326985c734aad8b71dcd03dcb4e46"
format_stamp: "Formatted at 2026-08-12 02:48:05 on dist-test-slave-ss4n"
server_key: "61cdb2fd1260d351cef0fe61ed5dd1ac"
server_key_iv: "be7ae9b998e43e6b238937d0971b3f4a"
server_key_version: "encryptionkey@0"
I20260812 02:48:05.185393 7586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:05.213976 7586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:05.215014 7586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:05.215476 7586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:05.216859 7586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:05.219027 7586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:05.219214 7586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:05.219365 7586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:05.219457 7586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:05.261909 7586 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:33359
I20260812 02:48:05.262085 7738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:33359 every 8 connection(s)
I20260812 02:48:05.263450 7586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:48:05.271889 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 7586
I20260812 02:48:05.272226 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:48:05.272754 18195 external_mini_cluster.cc:1428] Setting key 4be798d7384af97be4dad44bc777fb86
I20260812 02:48:05.276711 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37827
--builtin_ntp_servers=127.17.196.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:48:05.282577 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.266855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:59571 (local address 127.17.196.254:37827)
0812 02:48:05.267356 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:05.267361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:05.269097 (+ 1736us) server_negotiation.cc:407] Connection header received
0812 02:48:05.272196 (+ 3099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:05.272204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:05.272656 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:05.272878 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:05.275048 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.276214 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.278425 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.279205 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.279376 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.281249 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:05.281258 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:05.281266 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:05.281298 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:05.281358 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:05.281379 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:05.281516 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:05.281946 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 02:48:05.282096 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":247,"threads_started":1}
I20260812 02:48:05.283643 7742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.267477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37827 (local address 127.17.196.194:59571)
0812 02:48:05.268880 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:05.268926 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:05.271566 (+ 2640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:05.273350 (+ 1784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:05.273406 (+ 56us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:05.273639 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:05.274756 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.274789 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.276494 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.276498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.278224 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.278243 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.279534 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.279540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.279831 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.280781 (+ 950us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:48:05.280785 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:05.280838 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:48:05.280955 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:05.281605 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:05.281665 (+ 60us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:05.281678 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 02:48:05.281879 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 02:48:05.282287 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":459,"threads_started":1}
I20260812 02:48:05.285991 7739 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37827
I20260812 02:48:05.286618 7739 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:05.288084 7739 heartbeater.cc:511] Master 127.17.196.254:37827 requested a full tablet report, sending...
I20260812 02:48:05.290289 7361 ts_manager.cc:194] Registered new tserver with Master: 58e326985c734aad8b71dcd03dcb4e46 (127.17.196.194:33359)
I20260812 02:48:05.291265 7361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.194:59571
W20260812 02:48:05.722998 7743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:05.723387 7743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:05.723595 7743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:05.732519 7743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:05.732748 7743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:05.732940 7743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:48:05.744923 7743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:05.746600 7743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:05.748946 7743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:48:05.752729 7582 heartbeater.cc:503] Master 127.17.196.254:37827 was elected leader, sending a full tablet report...
W20260812 02:48:05.768127 7748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:05.768007 7751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:05.768174 7749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:05.769652 7743 server_base.cc:1034] running on GCE node
I20260812 02:48:05.770550 7743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:05.771842 7743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:05.773168 7743 hybrid_clock.cc:648] HybridClock initialized: now 1786502885773005 us; error 71 us; skew 500 ppm
I20260812 02:48:05.773717 7743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:48:05.777064 7743 webserver.cc:460] Webserver started at http://127.17.196.195:37391/ using document root <none> and password file <none>
I20260812 02:48:05.778190 7743 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:05.778316 7743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:05.778784 7743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:05.782968 7743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "381f1fb1266a40019f61de7f2339ac4a"
format_stamp: "Formatted at 2026-08-12 02:48:05 on dist-test-slave-ss4n"
server_key: "e0d04406b4bd948b8a6478ff7abb2d54"
server_key_iv: "785adff6d853424e2fce96a53bab1c9e"
server_key_version: "encryptionkey@0"
I20260812 02:48:05.783985 7743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "381f1fb1266a40019f61de7f2339ac4a"
format_stamp: "Formatted at 2026-08-12 02:48:05 on dist-test-slave-ss4n"
server_key: "e0d04406b4bd948b8a6478ff7abb2d54"
server_key_iv: "785adff6d853424e2fce96a53bab1c9e"
server_key_version: "encryptionkey@0"
I20260812 02:48:05.790815 7743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:48:05.795543 7758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:05.797349 7743 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 02:48:05.797614 7743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "381f1fb1266a40019f61de7f2339ac4a"
format_stamp: "Formatted at 2026-08-12 02:48:05 on dist-test-slave-ss4n"
server_key: "e0d04406b4bd948b8a6478ff7abb2d54"
server_key_iv: "785adff6d853424e2fce96a53bab1c9e"
server_key_version: "encryptionkey@0"
I20260812 02:48:05.797890 7743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:05.824703 7743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:05.825742 7743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:05.826097 7743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:05.827198 7743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:05.828837 7743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:05.828961 7743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:05.829105 7743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:05.829200 7743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:05.869071 7743 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:40813
I20260812 02:48:05.869127 7896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:40813 every 8 connection(s)
I20260812 02:48:05.870513 7743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:48:05.871004 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 7743
I20260812 02:48:05.871254 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:48:05.871915 18195 external_mini_cluster.cc:1428] Setting key cafa6e2c9e97bea1a04e52d55091077e
I20260812 02:48:05.885695 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.873774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:34589 (local address 127.17.196.254:37827)
0812 02:48:05.874502 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:05.874507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:05.876010 (+ 1503us) server_negotiation.cc:407] Connection header received
0812 02:48:05.878184 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:05.878190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:05.878296 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:05.878423 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:05.879686 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.880472 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.882301 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.882756 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.882911 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.884639 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:05.884644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:05.884647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:05.884679 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:05.884726 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:05.884739 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:05.884842 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:05.885100 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 02:48:05.885225 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":298,"threads_started":1}
I20260812 02:48:05.887101 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.874647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37827 (local address 127.17.196.195:34589)
0812 02:48:05.875846 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:05.875888 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:05.877822 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:05.878715 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:05.878725 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:05.878833 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:05.879486 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.879505 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.880658 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.880663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.882110 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.882126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.883031 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.883039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.883298 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.884200 (+ 902us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:48:05.884203 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:05.884252 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:48:05.884367 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:05.884844 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:05.884858 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:05.884870 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 02:48:05.885085 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 02:48:05.885703 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":330,"threads_started":1}
I20260812 02:48:05.888917 7897 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37827
I20260812 02:48:05.889320 7897 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:05.890242 7897 heartbeater.cc:511] Master 127.17.196.254:37827 requested a full tablet report, sending...
I20260812 02:48:05.891952 7361 ts_manager.cc:194] Registered new tserver with Master: 381f1fb1266a40019f61de7f2339ac4a (127.17.196.195:40813)
I20260812 02:48:05.892827 7361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.195:34589
I20260812 02:48:05.903226 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:48:05.918169 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.908711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.17.196.254:37827)
0812 02:48:05.909113 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:05.909124 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:05.909587 (+ 463us) server_negotiation.cc:407] Connection header received
0812 02:48:05.909728 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:05.909733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:05.909810 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:05.909944 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:05.910641 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.911686 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.913586 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.914382 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.914565 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.916788 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:05.916799 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:05.916806 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:05.916839 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:05.916894 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:05.916917 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:05.917151 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:05.917609 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 02:48:05.917694 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260812 02:48:05.924553 7361 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:48:05.932714 7361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:48:05.962970 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.950964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:44481 (local address 127.0.0.1:46312)
0812 02:48:05.952062 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:05.952102 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:05.952252 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:05.953542 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:05.953548 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:05.953600 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:05.955181 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.955196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.957057 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.957064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.959946 (+ 2882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.959970 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.961710 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.961718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.961968 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.962035 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 02:48:05.962243 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 02:48:05.962509 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":83,"thread_start_us":831,"threads_started":1}
I20260812 02:48:05.962953 7920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.951646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46312 (local address 127.17.196.193:44481)
0812 02:48:05.952543 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:05.952565 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:05.952611 (+ 46us) server_negotiation.cc:407] Connection header received
0812 02:48:05.952732 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:05.952740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:05.952979 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:05.953226 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:05.955395 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.956884 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.960200 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.961415 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.961726 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.962025 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:05.962308 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 02:48:05.962509 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":510,"threads_started":1}
I20260812 02:48:05.963559 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.950467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:33359 (local address 127.0.0.1:34028)
0812 02:48:05.951180 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:05.951204 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:05.951348 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:05.953434 (+ 2086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:05.953440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:05.953524 (+ 84us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:05.954490 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.954510 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.956542 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.956547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.959587 (+ 3040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.959607 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.962187 (+ 2580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.962196 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.962433 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.962469 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 02:48:05.962565 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 02:48:05.963247 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":587,"thread_start_us":355,"threads_started":1}
I20260812 02:48:05.964200 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.950911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.17.196.194:33359)
0812 02:48:05.952793 (+ 1882us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:05.952801 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:05.952829 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:48:05.952919 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:05.952926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:05.953171 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:05.953738 (+ 567us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:05.954720 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.956376 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.959885 (+ 3509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.961545 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.961780 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.962017 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:05.962785 (+ 768us) server_negotiation.cc:299] Negotiation successful
0812 02:48:05.962969 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1716,"thread_start_us":1568,"threads_started":1}
I20260812 02:48:05.963126 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.949000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40813 (local address 127.0.0.1:38406)
0812 02:48:05.950006 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:05.950052 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:05.950267 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:05.953193 (+ 2926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:05.953202 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:05.953236 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:05.954157 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.954282 (+ 125us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.956278 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.956287 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.959887 (+ 3600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:05.959905 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.962317 (+ 2412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.962328 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:05.962486 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.962521 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:48:05.962596 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 02:48:05.962732 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":179,"thread_start_us":920,"threads_started":1}
I20260812 02:48:05.965600 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:05.949559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38406 (local address 127.17.196.195:40813)
0812 02:48:05.951982 (+ 2423us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:05.952007 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:05.952054 (+ 47us) server_negotiation.cc:407] Connection header received
0812 02:48:05.952253 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:05.952262 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:05.952579 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:05.952805 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:05.954595 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.956117 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.960282 (+ 4165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:05.961772 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:05.961964 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:05.962186 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:05.964923 (+ 2737us) server_negotiation.cc:299] Negotiation successful
0812 02:48:05.965135 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2207,"thread_start_us":2009,"threads_started":1}
I20260812 02:48:05.968390 7506 tablet_service.cc:1467] Processing CreateTablet for tablet c6e590436a204669b7957533fa5e6e02 (DEFAULT_TABLE table=test-table [id=0115fbc49e8849ed838e17848c569644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:05.968393 7660 tablet_service.cc:1467] Processing CreateTablet for tablet c6e590436a204669b7957533fa5e6e02 (DEFAULT_TABLE table=test-table [id=0115fbc49e8849ed838e17848c569644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:05.968556 7816 tablet_service.cc:1467] Processing CreateTablet for tablet c6e590436a204669b7957533fa5e6e02 (DEFAULT_TABLE table=test-table [id=0115fbc49e8849ed838e17848c569644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:05.970134 7816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e590436a204669b7957533fa5e6e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:05.970148 7506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e590436a204669b7957533fa5e6e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:05.970144 7660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e590436a204669b7957533fa5e6e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:05.988397 7923 tablet_bootstrap.cc:492] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Bootstrap starting.
I20260812 02:48:05.993583 7923 tablet_bootstrap.cc:654] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:05.994517 7924 tablet_bootstrap.cc:492] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: Bootstrap starting.
I20260812 02:48:05.997097 7923 log.cc:824] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:05.998358 7925 tablet_bootstrap.cc:492] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: Bootstrap starting.
I20260812 02:48:05.999975 7924 tablet_bootstrap.cc:654] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:06.001850 7925 tablet_bootstrap.cc:654] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:06.002202 7924 log.cc:824] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:06.003285 7925 log.cc:824] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:06.005354 7923 tablet_bootstrap.cc:492] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: No bootstrap required, opened a new log
I20260812 02:48:06.005738 7923 ts_tablet_manager.cc:1397] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20260812 02:48:06.005888 7924 tablet_bootstrap.cc:492] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: No bootstrap required, opened a new log
I20260812 02:48:06.006181 7924 ts_tablet_manager.cc:1397] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.002s
I20260812 02:48:06.006551 7925 tablet_bootstrap.cc:492] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: No bootstrap required, opened a new log
I20260812 02:48:06.007238 7925 ts_tablet_manager.cc:1397] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.006s
I20260812 02:48:06.015905 7925 raft_consensus.cc:348] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.016392 7925 raft_consensus.cc:374] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:06.016505 7925 raft_consensus.cc:729] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e326985c734aad8b71dcd03dcb4e46, State: Initialized, Role: FOLLOWER
I20260812 02:48:06.017230 7925 consensus_queue.cc:260] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.018683 7739 heartbeater.cc:503] Master 127.17.196.254:37827 was elected leader, sending a full tablet report...
I20260812 02:48:06.019201 7925 ts_tablet_manager.cc:1428] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
I20260812 02:48:06.019302 7923 raft_consensus.cc:348] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.019894 7923 raft_consensus.cc:374] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:06.020004 7923 raft_consensus.cc:729] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 381f1fb1266a40019f61de7f2339ac4a, State: Initialized, Role: FOLLOWER
I20260812 02:48:06.022068 7924 raft_consensus.cc:348] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.022663 7924 raft_consensus.cc:374] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:06.022805 7924 raft_consensus.cc:729] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29acd0821ea54465bead0c3aa23e7a5a, State: Initialized, Role: FOLLOWER
I20260812 02:48:06.023600 7924 consensus_queue.cc:260] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.020804 7923 consensus_queue.cc:260] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.025205 7897 heartbeater.cc:503] Master 127.17.196.254:37827 was elected leader, sending a full tablet report...
I20260812 02:48:06.026315 7923 ts_tablet_manager.cc:1428] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Time spent starting tablet: real 0.020s user 0.011s sys 0.006s
I20260812 02:48:06.028353 7924 ts_tablet_manager.cc:1428] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: Time spent starting tablet: real 0.022s user 0.012s sys 0.005s
I20260812 02:48:06.064312 7930 raft_consensus.cc:482] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:48:06.064839 7930 raft_consensus.cc:504] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.067060 7930 leader_election.cc:288] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 381f1fb1266a40019f61de7f2339ac4a (127.17.196.195:40813), 58e326985c734aad8b71dcd03dcb4e46 (127.17.196.194:33359)
I20260812 02:48:06.076557 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:06.067774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:40727 (local address 127.17.196.195:40813)
0812 02:48:06.068079 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:06.068089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:06.069251 (+ 1162us) server_negotiation.cc:407] Connection header received
0812 02:48:06.069543 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:06.069549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:06.069616 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:06.069738 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:06.070639 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.071867 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.074604 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.075459 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.075697 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:06.075789 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:06.076135 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 02:48:06.076276 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260812 02:48:06.076632 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:06.068461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:33399 (local address 127.17.196.194:33359)
0812 02:48:06.068716 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:06.068722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:06.069448 (+ 726us) server_negotiation.cc:407] Connection header received
0812 02:48:06.069748 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:06.069754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:06.069856 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:06.069982 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:06.070774 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.071867 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.074567 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.075462 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.075697 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:06.075786 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:06.076142 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 02:48:06.076298 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:48:06.076701 7932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:06.067711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:33359 (local address 127.17.196.193:33399)
0812 02:48:06.069331 (+ 1620us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:06.069348 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:06.069579 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:06.070016 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:06.070021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:06.070055 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:06.070515 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:06.070527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.072051 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.072057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:06.074306 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:06.074324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.075737 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.075745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:06.075937 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:06.075981 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 02:48:06.076101 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 02:48:06.076271 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1488,"mutex_wait_us":188,"thread_start_us":839,"threads_started":1}
I20260812 02:48:06.077047 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:06.067623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40813 (local address 127.17.196.193:40727)
0812 02:48:06.069093 (+ 1470us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:06.069116 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:06.069334 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:06.069940 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:06.069947 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:06.069979 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:06.070447 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:06.070463 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.072105 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.072109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:06.074327 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:06.074344 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.075697 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.075705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:06.075935 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:06.075978 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 02:48:06.076102 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 02:48:06.076354 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1267,"mutex_wait_us":111,"thread_start_us":1350,"threads_started":1}
I20260812 02:48:06.078047 7684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e590436a204669b7957533fa5e6e02" candidate_uuid: "29acd0821ea54465bead0c3aa23e7a5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58e326985c734aad8b71dcd03dcb4e46" is_pre_election: true
I20260812 02:48:06.078262 7844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e590436a204669b7957533fa5e6e02" candidate_uuid: "29acd0821ea54465bead0c3aa23e7a5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "381f1fb1266a40019f61de7f2339ac4a" is_pre_election: true
I20260812 02:48:06.078755 7684 raft_consensus.cc:2454] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29acd0821ea54465bead0c3aa23e7a5a in term 0.
I20260812 02:48:06.078799 7844 raft_consensus.cc:2454] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29acd0821ea54465bead0c3aa23e7a5a in term 0.
I20260812 02:48:06.079555 7448 leader_election.cc:302] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29acd0821ea54465bead0c3aa23e7a5a, 58e326985c734aad8b71dcd03dcb4e46; no voters:
I20260812 02:48:06.079993 7930 raft_consensus.cc:2781] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:48:06.080175 7930 raft_consensus.cc:482] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:48:06.080310 7930 raft_consensus.cc:3037] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:06.082892 7930 raft_consensus.cc:504] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.083671 7930 leader_election.cc:288] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [CANDIDATE]: Term 1 election: Requested vote from peers 381f1fb1266a40019f61de7f2339ac4a (127.17.196.195:40813), 58e326985c734aad8b71dcd03dcb4e46 (127.17.196.194:33359)
I20260812 02:48:06.084542 7844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e590436a204669b7957533fa5e6e02" candidate_uuid: "29acd0821ea54465bead0c3aa23e7a5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "381f1fb1266a40019f61de7f2339ac4a"
I20260812 02:48:06.084699 7684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e590436a204669b7957533fa5e6e02" candidate_uuid: "29acd0821ea54465bead0c3aa23e7a5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58e326985c734aad8b71dcd03dcb4e46"
I20260812 02:48:06.084825 7844 raft_consensus.cc:3037] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:06.084981 7684 raft_consensus.cc:3037] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:06.087517 7684 raft_consensus.cc:2454] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29acd0821ea54465bead0c3aa23e7a5a in term 1.
I20260812 02:48:06.087590 7844 raft_consensus.cc:2454] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29acd0821ea54465bead0c3aa23e7a5a in term 1.
I20260812 02:48:06.088121 7448 leader_election.cc:302] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29acd0821ea54465bead0c3aa23e7a5a, 58e326985c734aad8b71dcd03dcb4e46; no voters:
I20260812 02:48:06.088672 7930 raft_consensus.cc:2781] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:06.089272 7930 raft_consensus.cc:686] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 1 LEADER]: Becoming Leader. State: Replica: 29acd0821ea54465bead0c3aa23e7a5a, State: Running, Role: LEADER
I20260812 02:48:06.090013 7930 consensus_queue.cc:237] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } }
I20260812 02:48:06.095396 7361 catalog_manager.cc:5495] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 29acd0821ea54465bead0c3aa23e7a5a (127.17.196.193). New cstate: current_term: 1 leader_uuid: "29acd0821ea54465bead0c3aa23e7a5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29acd0821ea54465bead0c3aa23e7a5a" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 44481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 } health_report { overall_health: UNKNOWN } } }
W20260812 02:48:06.122592 7899 tablet.cc:2369] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:48:06.129220 7920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:06.122588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.17.196.193:44481)
0812 02:48:06.122880 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:06.122889 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:06.122915 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:48:06.123165 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:06.123169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:06.123266 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:06.123404 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:48:06.123922 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.124810 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.126327 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.126822 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.126990 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:06.127676 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:48:06.128300 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:48:06.128458 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:06.128818 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 02:48:06.128879 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 02:48:06.139156 7684 raft_consensus.cc:1261] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 1 FOLLOWER]: Refusing update from remote peer 29acd0821ea54465bead0c3aa23e7a5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:48:06.139168 7844 raft_consensus.cc:1261] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 1 FOLLOWER]: Refusing update from remote peer 29acd0821ea54465bead0c3aa23e7a5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:48:06.140329 7930 consensus_queue.cc:1035] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58e326985c734aad8b71dcd03dcb4e46" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 33359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:48:06.140689 7934 consensus_queue.cc:1035] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "381f1fb1266a40019f61de7f2339ac4a" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:48:06.156840 7939 mvcc.cc:205] Tried to move back new op lower bound from 7317515821613240320 to 7317515821429489664. Current Snapshot: MvccSnapshot[applied={T|T < 7317515821613240320}]
I20260812 02:48:06.171689 7361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:48:06.172050 7361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:48:06.176515 7361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 113ba9b6e2214a9d92dfa97a32f21fd4: Sending DeleteTablet for 3 replicas of tablet c6e590436a204669b7957533fa5e6e02
I20260812 02:48:06.178356 7660 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e590436a204669b7957533fa5e6e02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:06 UTC) from {username='slave'} at 127.0.0.1:34028
I20260812 02:48:06.178680 7816 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e590436a204669b7957533fa5e6e02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:06 UTC) from {username='slave'} at 127.0.0.1:38406
I20260812 02:48:06.179570 7506 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e590436a204669b7957533fa5e6e02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:06 UTC) from {username='slave'} at 127.0.0.1:46312
I20260812 02:48:06.181149 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 7432
I20260812 02:48:06.186827 7952 tablet_replica.cc:330] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: stopping tablet replica
I20260812 02:48:06.187384 7952 raft_consensus.cc:2229] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:48:06.188177 7952 raft_consensus.cc:2258] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:06.189116 7951 tablet_replica.cc:330] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: stopping tablet replica
I20260812 02:48:06.189683 7951 raft_consensus.cc:2229] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:48:06.190263 7951 raft_consensus.cc:2258] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:06.193324 7952 ts_tablet_manager.cc:1905] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:06.203976 7951 ts_tablet_manager.cc:1905] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:06.204312 7950 tablet_replica.cc:330] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: stopping tablet replica
I20260812 02:48:06.204846 7950 raft_consensus.cc:2229] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:48:06.205410 7950 raft_consensus.cc:2258] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:06.206364 7952 ts_tablet_manager.cc:1918] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:06.206588 7952 log.cc:1196] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/c6e590436a204669b7957533fa5e6e02
I20260812 02:48:06.207255 7952 ts_tablet_manager.cc:1939] T c6e590436a204669b7957533fa5e6e02 P 29acd0821ea54465bead0c3aa23e7a5a: Deleting consensus metadata
I20260812 02:48:06.208781 7950 ts_tablet_manager.cc:1905] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:06.210032 7920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:06.186016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46332 (local address 127.17.196.193:44481)
0812 02:48:06.186228 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:06.186236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:06.198646 (+ 12410us) server_negotiation.cc:407] Connection header received
0812 02:48:06.199406 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:06.199415 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:06.199616 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:06.199719 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:06.200658 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.202331 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.203904 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.204626 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.204782 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:06.206984 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:06.207001 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:06.207021 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:06.207972 (+ 951us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:06.208055 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:06.208101 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:06.208332 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:06.208829 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 02:48:06.209101 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:48:06.209970 7344 catalog_manager.cc:4841] TS 29acd0821ea54465bead0c3aa23e7a5a (127.17.196.193:44481): tablet c6e590436a204669b7957533fa5e6e02 (table test-table [id=0115fbc49e8849ed838e17848c569644]) successfully deleted
I20260812 02:48:06.211645 7577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:06.215592 7950 ts_tablet_manager.cc:1918] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:06.215906 7950 log.cc:1196] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/c6e590436a204669b7957533fa5e6e02
I20260812 02:48:06.216559 7950 ts_tablet_manager.cc:1939] T c6e590436a204669b7957533fa5e6e02 P 58e326985c734aad8b71dcd03dcb4e46: Deleting consensus metadata
I20260812 02:48:06.219475 7342 catalog_manager.cc:4841] TS 58e326985c734aad8b71dcd03dcb4e46 (127.17.196.194:33359): tablet c6e590436a204669b7957533fa5e6e02 (table test-table [id=0115fbc49e8849ed838e17848c569644]) successfully deleted
I20260812 02:48:06.219789 7951 ts_tablet_manager.cc:1918] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:06.220038 7951 log.cc:1196] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/c6e590436a204669b7957533fa5e6e02
I20260812 02:48:06.220786 7951 ts_tablet_manager.cc:1939] T c6e590436a204669b7957533fa5e6e02 P 381f1fb1266a40019f61de7f2339ac4a: Deleting consensus metadata
I20260812 02:48:06.222714 7344 catalog_manager.cc:4841] TS 381f1fb1266a40019f61de7f2339ac4a (127.17.196.195:40813): tablet c6e590436a204669b7957533fa5e6e02 (table test-table [id=0115fbc49e8849ed838e17848c569644]) successfully deleted
I20260812 02:48:06.323320 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 7432
I20260812 02:48:06.348228 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 7586
I20260812 02:48:06.355937 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:06.349189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.17.196.194:33359)
0812 02:48:06.349429 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:06.349436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:06.349457 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:48:06.349746 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:06.349753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:06.349851 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:06.349961 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:06.350404 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.351236 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.352519 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.352963 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.353104 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:06.354434 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:06.354444 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:06.354459 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:06.355082 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:06.355133 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:06.355156 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:06.355286 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:06.355428 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 02:48:06.355565 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 02:48:06.356765 7734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:06.455868 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 7586
I20260812 02:48:06.480827 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 7743
I20260812 02:48:06.489902 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:06.481893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38410 (local address 127.17.196.195:40813)
0812 02:48:06.482120 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:06.482126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:06.482512 (+ 386us) server_negotiation.cc:407] Connection header received
0812 02:48:06.482605 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:06.482609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:06.482708 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:06.482828 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:06.483344 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.484307 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.486148 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:06.486688 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:06.486811 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:06.488069 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:06.488080 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:06.488097 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:06.488835 (+ 738us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:06.488891 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:06.488921 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:06.489048 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:06.489232 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 02:48:06.489350 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:48:06.490782 7891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:06.586477 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 7743
I20260812 02:48:06.610172 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 7319
I20260812 02:48:06.611521 7396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:06.729640 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 7319
2026-08-12T02:48:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3303 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:06 dist-test-slave-ss4n krb5kdc[7965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:06 dist-test-slave-ss4n krb5kdc[7965](info): set up 2 sockets
Aug 12 02:48:06 dist-test-slave-ss4n krb5kdc[7965](info): commencing operation
krb5kdc: starting...
W20260812 02:48:08.076521 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:08 dist-test-slave-ss4n krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:48:08.468693 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37619
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:37619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:08.894922 7979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:08.895283 7979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:08.895476 7979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:08.904449 7979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:08.904611 7979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:08.904716 7979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:08.904842 7979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:08.904973 7979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:08.916698 7979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:37619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:37619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:08.918258 7979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:08.919803 7979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:08.936080 7989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:08.936219 7986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:08.936240 7985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:08.939067 7979 server_base.cc:1034] running on GCE node
I20260812 02:48:08.940331 7979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:08.942334 7979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:08.943646 7979 hybrid_clock.cc:648] HybridClock initialized: now 1786502888943590 us; error 46 us; skew 500 ppm
I20260812 02:48:08.944218 7979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:08 dist-test-slave-ss4n krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:08.953645 7979 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:48:08.955818 7979 webserver.cc:460] Webserver started at http://127.17.196.254:35245/ using document root <none> and password file <none>
I20260812 02:48:08.956759 7979 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:08.956897 7979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:08.957288 7979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:08.961062 7979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "efa3116999a449c4b6be834e853c019e"
format_stamp: "Formatted at 2026-08-12 02:48:08 on dist-test-slave-ss4n"
server_key: "7be41f2af75424526b45378e46c78019"
server_key_iv: "703c01ecd134cd2efe0e29ed4b0c3f7a"
server_key_version: "encryptionkey@0"
I20260812 02:48:08.962121 7979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "efa3116999a449c4b6be834e853c019e"
format_stamp: "Formatted at 2026-08-12 02:48:08 on dist-test-slave-ss4n"
server_key: "7be41f2af75424526b45378e46c78019"
server_key_iv: "703c01ecd134cd2efe0e29ed4b0c3f7a"
server_key_version: "encryptionkey@0"
I20260812 02:48:08.968957 7979 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 02:48:08.973940 7997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:08.975808 7979 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:48:08.976097 7979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "efa3116999a449c4b6be834e853c019e"
format_stamp: "Formatted at 2026-08-12 02:48:08 on dist-test-slave-ss4n"
server_key: "7be41f2af75424526b45378e46c78019"
server_key_iv: "703c01ecd134cd2efe0e29ed4b0c3f7a"
server_key_version: "encryptionkey@0"
I20260812 02:48:08.976366 7979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:08.999895 7979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:09.005928 7979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:09.006314 7979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:09.031461 7979 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:37619
I20260812 02:48:09.031523 8059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:37619 every 8 connection(s)
I20260812 02:48:09.033070 7979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:09.037992 8060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:09.039702 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 7979
I20260812 02:48:09.039947 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:09.040597 18195 external_mini_cluster.cc:1428] Setting key 51ce3500dd7e0e78416f1da46cedaa33
I20260812 02:48:09.054596 8060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e: Bootstrap starting.
Aug 12 02:48:09 dist-test-slave-ss4n krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:09.059221 8060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:09.060582 8060 log.cc:824] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:09.064440 8060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e: No bootstrap required, opened a new log
I20260812 02:48:09.070492 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:09.043329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35044 (local address 127.17.196.254:37619)
0812 02:48:09.044239 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:09.044252 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:09.044300 (+ 48us) server_negotiation.cc:407] Connection header received
0812 02:48:09.046297 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:09.046333 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:09.046875 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:09.047407 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:09.049251 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:09.051498 (+ 2247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:09.053273 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:09.054239 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:09.054488 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:09.060535 (+ 6047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:09.060546 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:09.060570 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:09.060643 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:09.064733 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:09.065852 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:09.065862 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:09.065871 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:09.066140 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:09.066689 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:09.066694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:09.066696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:09.067417 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:09.067840 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:09.068419 (+ 579us) server_negotiation.cc:299] Negotiation successful
0812 02:48:09.068985 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":408,"threads_started":1}
I20260812 02:48:09.074542 8060 raft_consensus.cc:348] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa3116999a449c4b6be834e853c019e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37619 } }
I20260812 02:48:09.075039 8060 raft_consensus.cc:374] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:09.075179 8060 raft_consensus.cc:729] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa3116999a449c4b6be834e853c019e, State: Initialized, Role: FOLLOWER
I20260812 02:48:09.075856 8060 consensus_queue.cc:260] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa3116999a449c4b6be834e853c019e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37619 } }
I20260812 02:48:09.076149 8060 raft_consensus.cc:388] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:09.076335 8060 raft_consensus.cc:482] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:09.076543 8060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:09.079563 8060 raft_consensus.cc:504] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa3116999a449c4b6be834e853c019e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37619 } }
I20260812 02:48:09.080248 8060 leader_election.cc:302] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efa3116999a449c4b6be834e853c019e; no voters:
I20260812 02:48:09.081059 8060 leader_election.cc:288] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:09.081349 8066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:09.082784 8066 raft_consensus.cc:686] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [term 1 LEADER]: Becoming Leader. State: Replica: efa3116999a449c4b6be834e853c019e, State: Running, Role: LEADER
I20260812 02:48:09.083571 8066 consensus_queue.cc:237] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa3116999a449c4b6be834e853c019e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37619 } }
I20260812 02:48:09.084157 8060 sys_catalog.cc:564] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:09.087283 8069 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [sys.catalog]: SysCatalogTable state changed. Reason: New leader efa3116999a449c4b6be834e853c019e. Latest consensus state: current_term: 1 leader_uuid: "efa3116999a449c4b6be834e853c019e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa3116999a449c4b6be834e853c019e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37619 } } }
I20260812 02:48:09.087883 8069 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:09.087267 8068 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efa3116999a449c4b6be834e853c019e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa3116999a449c4b6be834e853c019e" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 37619 } } }
I20260812 02:48:09.088577 8068 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:09.091109 8078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:09.098769 8078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:09.111943 8078 catalog_manager.cc:1347] Generated new cluster ID: 9ebddeb1f71a4fda9434383884ed2bc6
I20260812 02:48:09.112103 8078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:09.124014 8078 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:09.126053 8078 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:09.141952 8078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e: Generated new TSK 0
I20260812 02:48:09.143118 8078 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
Entry for principal kudu/127.17.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab.
I20260812 02:48:09.235811 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37619
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:09.657238 8094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:09.657706 8094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:09.657879 8094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:09.666352 8094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:09.666549 8094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:09.666731 8094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:48:09.677806 8094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.193.keytab
--principal=kudu/127.17.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:09.679343 8094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:09.680857 8094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:09.697191 8103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:09.697322 8101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:09.697858 8100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:09.698264 8094 server_base.cc:1034] running on GCE node
I20260812 02:48:09.699049 8094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:09.700280 8094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:09.701737 8094 hybrid_clock.cc:648] HybridClock initialized: now 1786502889701669 us; error 61 us; skew 500 ppm
I20260812 02:48:09.702214 8094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:09 dist-test-slave-ss4n krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502889, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:09.706571 8094 init.cc:377] Logged in from keytab as kudu/127.17.196.193@KRBTEST.COM (short username kudu)
I20260812 02:48:09.708513 8094 webserver.cc:460] Webserver started at http://127.17.196.193:32903/ using document root <none> and password file <none>
I20260812 02:48:09.709336 8094 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:09.709470 8094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:09.709898 8094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:09.713392 8094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "e15108b6f3984e28a7a580ef910dc06c"
format_stamp: "Formatted at 2026-08-12 02:48:09 on dist-test-slave-ss4n"
server_key: "c89b5b9c9aad979ecf9033bb7f74a67e"
server_key_iv: "7634575e23cab55bf1856208bdb7e529"
server_key_version: "encryptionkey@0"
I20260812 02:48:09.714381 8094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "e15108b6f3984e28a7a580ef910dc06c"
format_stamp: "Formatted at 2026-08-12 02:48:09 on dist-test-slave-ss4n"
server_key: "c89b5b9c9aad979ecf9033bb7f74a67e"
server_key_iv: "7634575e23cab55bf1856208bdb7e529"
server_key_version: "encryptionkey@0"
I20260812 02:48:09.721258 8094 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:48:09.726019 8111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:09.727921 8094 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:48:09.728196 8094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "e15108b6f3984e28a7a580ef910dc06c"
format_stamp: "Formatted at 2026-08-12 02:48:09 on dist-test-slave-ss4n"
server_key: "c89b5b9c9aad979ecf9033bb7f74a67e"
server_key_iv: "7634575e23cab55bf1856208bdb7e529"
server_key_version: "encryptionkey@0"
I20260812 02:48:09.728464 8094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:09.750620 8094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:09.756351 8094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:09.756755 8094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:09.757977 8094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:09.759622 8094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:09.759704 8094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:09.759847 8094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:09.759912 8094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:09.799165 8094 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:37095
I20260812 02:48:09.799258 8246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:37095 every 8 connection(s)
I20260812 02:48:09.800664 8094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:48:09.808238 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 8094
I20260812 02:48:09.808475 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:48:09.809094 18195 external_mini_cluster.cc:1428] Setting key e2b171b6b087bdb4e5ba1991555e8c54
Aug 12 02:48:09 dist-test-slave-ss4n krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502889, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.193@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:09.824736 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:09.803651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:49185 (local address 127.17.196.254:37619)
0812 02:48:09.804167 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:09.804172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:09.805015 (+ 843us) server_negotiation.cc:407] Connection header received
0812 02:48:09.807203 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:09.807209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:09.807317 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:09.807458 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:09.810114 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:09.811142 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:09.813245 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:09.813755 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:09.813912 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:09.817903 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:09.817909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:09.817912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:09.817952 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:09.820983 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:09.822065 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:09.822076 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:09.822078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:09.822232 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:09.822762 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:09.822766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:09.822769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:09.823067 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:09.823237 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:09.824057 (+ 820us) server_negotiation.cc:299] Negotiation successful
0812 02:48:09.824335 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":276,"threads_started":1}
I20260812 02:48:09.826033 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:09.804062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37619 (local address 127.17.196.193:49185)
0812 02:48:09.804817 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:09.804865 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:09.806872 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:09.807823 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:09.807840 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:09.808774 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:09.809826 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:09.809863 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:09.811493 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:09.811499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:09.813034 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:09.813051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:09.814050 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:09.814058 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:09.814323 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:09.815109 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:48:09.815140 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:09.817612 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:09.821210 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:09.821222 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:09.821253 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:09.821899 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:09.822366 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:09.822372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:09.822375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:09.822603 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:09.823213 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:09.823222 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:09.823633 (+ 411us) client_negotiation.cc:769] Sending connection context
0812 02:48:09.823930 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 02:48:09.824614 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":300,"threads_started":1}
I20260812 02:48:09.827914 8247 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37619
I20260812 02:48:09.828333 8247 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:09.829372 8247 heartbeater.cc:511] Master 127.17.196.254:37619 requested a full tablet report, sending...
I20260812 02:48:09.832393 8019 ts_manager.cc:194] Registered new tserver with Master: e15108b6f3984e28a7a580ef910dc06c (127.17.196.193:37095)
I20260812 02:48:09.834471 8019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.193@KRBTEST.COM'} at 127.17.196.193:49185
WARNING: no policy specified for kudu/127.17.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
Entry for principal kudu/127.17.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab.
I20260812 02:48:09.895043 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37619
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:10.291103 8254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:10.291419 8254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:10.291591 8254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:10.300015 8254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:10.300160 8254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:10.300362 8254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:48:10.311174 8254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.194.keytab
--principal=kudu/127.17.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:10.312659 8254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:10.314167 8254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:10.330155 8259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:10.330066 8260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:10.330758 8262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:10.332223 8254 server_base.cc:1034] running on GCE node
I20260812 02:48:10.333029 8254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:10.334332 8254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:10.335626 8254 hybrid_clock.cc:648] HybridClock initialized: now 1786502890335557 us; error 128 us; skew 500 ppm
I20260812 02:48:10.336067 8254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:10 dist-test-slave-ss4n krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502890, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:10.340224 8254 init.cc:377] Logged in from keytab as kudu/127.17.196.194@KRBTEST.COM (short username kudu)
I20260812 02:48:10.342192 8254 webserver.cc:460] Webserver started at http://127.17.196.194:45617/ using document root <none> and password file <none>
I20260812 02:48:10.342962 8254 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:10.343122 8254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:10.343482 8254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:10.346776 8254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "6eeff1c07ad841128ef712056bb2d241"
format_stamp: "Formatted at 2026-08-12 02:48:10 on dist-test-slave-ss4n"
server_key: "3ad3b0f8ccaaeafa110bfb6c422af014"
server_key_iv: "1ddef86259cce12a427c683cf2c38ad5"
server_key_version: "encryptionkey@0"
I20260812 02:48:10.347681 8254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "6eeff1c07ad841128ef712056bb2d241"
format_stamp: "Formatted at 2026-08-12 02:48:10 on dist-test-slave-ss4n"
server_key: "3ad3b0f8ccaaeafa110bfb6c422af014"
server_key_iv: "1ddef86259cce12a427c683cf2c38ad5"
server_key_version: "encryptionkey@0"
I20260812 02:48:10.353978 8254 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:48:10.358829 8270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:10.360532 8254 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:48:10.360778 8254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "6eeff1c07ad841128ef712056bb2d241"
format_stamp: "Formatted at 2026-08-12 02:48:10 on dist-test-slave-ss4n"
server_key: "3ad3b0f8ccaaeafa110bfb6c422af014"
server_key_iv: "1ddef86259cce12a427c683cf2c38ad5"
server_key_version: "encryptionkey@0"
I20260812 02:48:10.361035 8254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:10.380383 8254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:10.385807 8254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:10.386142 8254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:10.387269 8254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:10.388911 8254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:10.389025 8254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:10.389182 8254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:10.389261 8254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:10.428007 8254 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:40429
I20260812 02:48:10.428114 8400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:40429 every 8 connection(s)
I20260812 02:48:10.429541 8254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:48:10.439883 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 8254
I20260812 02:48:10.440059 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:48:10.440536 18195 external_mini_cluster.cc:1428] Setting key 10f99ad2e680c0d03b21d1466800da3e
Aug 12 02:48:10 dist-test-slave-ss4n krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502890, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.194@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:10.455309 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:10.432581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:44739 (local address 127.17.196.254:37619)
0812 02:48:10.433199 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:10.433204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:10.434475 (+ 1271us) server_negotiation.cc:407] Connection header received
0812 02:48:10.436548 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:10.436555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:10.436662 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:10.436796 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:10.438703 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:10.439608 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:10.442100 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:10.442800 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:10.442902 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:10.447009 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:10.447018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:10.447026 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:10.447080 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:10.451091 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:10.452251 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:10.452260 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:10.452266 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:10.452446 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:10.452992 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:10.453000 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:10.453006 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:10.453452 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:10.453665 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:10.454489 (+ 824us) server_negotiation.cc:299] Negotiation successful
0812 02:48:10.454804 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":315,"threads_started":1}
I20260812 02:48:10.456017 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:10.433240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37619 (local address 127.17.196.194:44739)
0812 02:48:10.434278 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:10.434322 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:10.436206 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:10.437085 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:10.437094 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:10.437834 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:10.438470 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:10.438490 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:10.439865 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:10.439871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:10.441895 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:10.441915 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:10.443102 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:10.443110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:10.443338 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:10.444111 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:48:10.444138 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:10.446652 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:10.451337 (+ 4685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:10.451348 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:10.451366 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:10.452053 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:10.452611 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:10.452615 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:10.452618 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:10.452843 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:10.453664 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:10.453672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:10.454055 (+ 383us) client_negotiation.cc:769] Sending connection context
0812 02:48:10.454322 (+ 267us) client_negotiation.cc:240] Negotiation successful
0812 02:48:10.454778 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":385,"threads_started":1}
I20260812 02:48:10.458118 8401 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37619
I20260812 02:48:10.458504 8401 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:10.459498 8401 heartbeater.cc:511] Master 127.17.196.254:37619 requested a full tablet report, sending...
I20260812 02:48:10.461560 8019 ts_manager.cc:194] Registered new tserver with Master: 6eeff1c07ad841128ef712056bb2d241 (127.17.196.194:40429)
I20260812 02:48:10.463368 8019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.194@KRBTEST.COM'} at 127.17.196.194:44739
WARNING: no policy specified for kudu/127.17.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
Entry for principal kudu/127.17.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab.
I20260812 02:48:10.527235 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37619
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:48:10.839361 8247 heartbeater.cc:503] Master 127.17.196.254:37619 was elected leader, sending a full tablet report...
W20260812 02:48:10.920593 8407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:10.920974 8407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:10.921090 8407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:10.930518 8407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:10.930677 8407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:10.930817 8407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:48:10.943197 8407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.195.keytab
--principal=kudu/127.17.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:37619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:10.944658 8407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:10.946285 8407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:10.962594 8417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:10.962678 8415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:10.962932 8414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:10.964182 8407 server_base.cc:1034] running on GCE node
I20260812 02:48:10.964917 8407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:10.966317 8407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:10.967581 8407 hybrid_clock.cc:648] HybridClock initialized: now 1786502890967504 us; error 72 us; skew 500 ppm
I20260812 02:48:10.967922 8407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:10 dist-test-slave-ss4n krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502890, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:10.972090 8407 init.cc:377] Logged in from keytab as kudu/127.17.196.195@KRBTEST.COM (short username kudu)
I20260812 02:48:10.973984 8407 webserver.cc:460] Webserver started at http://127.17.196.195:40929/ using document root <none> and password file <none>
I20260812 02:48:10.974776 8407 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:10.974896 8407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:10.975211 8407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:10.978829 8407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "e12b49051d43463e8704f391c6fcc8af"
format_stamp: "Formatted at 2026-08-12 02:48:10 on dist-test-slave-ss4n"
server_key: "7ded849a9d543c8b183d0cbaae48fc42"
server_key_iv: "92f2c792717bfc55816103372483928e"
server_key_version: "encryptionkey@0"
I20260812 02:48:10.979734 8407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "e12b49051d43463e8704f391c6fcc8af"
format_stamp: "Formatted at 2026-08-12 02:48:10 on dist-test-slave-ss4n"
server_key: "7ded849a9d543c8b183d0cbaae48fc42"
server_key_iv: "92f2c792717bfc55816103372483928e"
server_key_version: "encryptionkey@0"
I20260812 02:48:10.986608 8407 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 02:48:10.991261 8426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:10.993055 8407 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:48:10.993317 8407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "e12b49051d43463e8704f391c6fcc8af"
format_stamp: "Formatted at 2026-08-12 02:48:10 on dist-test-slave-ss4n"
server_key: "7ded849a9d543c8b183d0cbaae48fc42"
server_key_iv: "92f2c792717bfc55816103372483928e"
server_key_version: "encryptionkey@0"
I20260812 02:48:10.993686 8407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:11.015012 8407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:11.020629 8407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:11.020990 8407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:11.022285 8407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:11.024094 8407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:11.024183 8407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:11.024338 8407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:11.024396 8407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:11.066349 8407 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:33705
I20260812 02:48:11.066403 8556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:33705 every 8 connection(s)
I20260812 02:48:11.067909 8407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:48:11.068511 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 8407
I20260812 02:48:11.068975 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:48:11.069769 18195 external_mini_cluster.cc:1428] Setting key 57c7aeb0b77e16a1321726908462d668
Aug 12 02:48:11 dist-test-slave-ss4n krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502890, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.195@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:11.095311 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.071633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:48811 (local address 127.17.196.254:37619)
0812 02:48:11.072232 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.072238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.074116 (+ 1878us) server_negotiation.cc:407] Connection header received
0812 02:48:11.076336 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.076342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.076425 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.076538 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:11.079152 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.080200 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.082396 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.082902 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.083028 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.087939 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:11.087945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:11.087949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:11.087989 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:11.091465 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.092451 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.092456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.092459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.092613 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.093197 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.093204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.093209 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.093661 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:11.093806 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.094764 (+ 958us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.094971 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":291,"threads_started":1}
I20260812 02:48:11.096457 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.072655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:37619 (local address 127.17.196.195:48811)
0812 02:48:11.073879 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:11.073967 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:11.075935 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:11.076940 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:11.076950 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:11.077988 (+ 1038us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:11.078883 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.078908 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.080418 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.080423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.082192 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.082211 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.083171 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.083177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.083397 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.084239 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:48:11.084279 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:11.087621 (+ 3342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:11.091669 (+ 4048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:11.091679 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:11.091697 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:11.092291 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:11.092762 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:48:11.092766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:48:11.092769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:48:11.093024 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:48:11.093841 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:11.093849 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:11.094302 (+ 453us) client_negotiation.cc:769] Sending connection context
0812 02:48:11.094603 (+ 301us) client_negotiation.cc:240] Negotiation successful
0812 02:48:11.095097 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":463,"threads_started":1}
I20260812 02:48:11.098428 8557 heartbeater.cc:347] Connected to a master server at 127.17.196.254:37619
I20260812 02:48:11.098855 8557 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:11.099956 8557 heartbeater.cc:511] Master 127.17.196.254:37619 requested a full tablet report, sending...
I20260812 02:48:11.102067 8019 ts_manager.cc:194] Registered new tserver with Master: e12b49051d43463e8704f391c6fcc8af (127.17.196.195:33705)
I20260812 02:48:11.103156 8019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.196.195@KRBTEST.COM'} at 127.17.196.195:48811
I20260812 02:48:11.113229 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:48:11.136006 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.118733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.17.196.254:37619)
0812 02:48:11.119086 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.119095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.119440 (+ 345us) server_negotiation.cc:407] Connection header received
0812 02:48:11.119760 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.119766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.119861 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.120004 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:11.121849 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.122727 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.124429 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.124964 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.125152 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.128279 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:11.128284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:11.128288 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:11.128332 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:11.131927 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.133006 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.133018 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.133022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.133237 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.133927 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.133931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.133933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.134378 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:11.134580 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.135341 (+ 761us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.135705 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 02:48:11.141072 8019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:48:11.148617 8019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:48:11.182580 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.165507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.17.196.193:37095)
0812 02:48:11.170350 (+ 4843us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.170366 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.170860 (+ 494us) server_negotiation.cc:407] Connection header received
0812 02:48:11.170988 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.170996 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.171395 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.171662 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:11.172967 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.175335 (+ 2368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.179376 (+ 4041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.180930 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.181240 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.181461 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.181958 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.182188 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4555,"thread_start_us":4285,"threads_started":1}
I20260812 02:48:11.183574 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.164691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:37095 (local address 127.0.0.1:39196)
0812 02:48:11.170524 (+ 5833us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:11.170548 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:11.170713 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:11.172089 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:11.172102 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:11.172158 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:11.172707 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.172725 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.175572 (+ 2847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.175582 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.179058 (+ 3476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.179088 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.181223 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.181231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.181535 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.181598 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 02:48:11.182880 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0812 02:48:11.183136 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5693,"thread_start_us":1058,"threads_started":1}
I20260812 02:48:11.185945 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.164051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:40429 (local address 127.0.0.1:35950)
0812 02:48:11.165819 (+ 1768us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:11.165893 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:11.166184 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:11.173428 (+ 7244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:11.173435 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:11.173501 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:11.174157 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.174177 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.179516 (+ 5339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.179526 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.182802 (+ 3276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.182818 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.185030 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.185040 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.185238 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.185285 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 02:48:11.185368 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 02:48:11.185554 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1274,"thread_start_us":6335,"threads_started":1}
I20260812 02:48:11.186651 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.164331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.17.196.194:40429)
0812 02:48:11.170882 (+ 6551us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.170889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.170961 (+ 72us) server_negotiation.cc:407] Connection header received
0812 02:48:11.172737 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.172754 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.173104 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.174475 (+ 1371us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:11.174794 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.179263 (+ 4469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.183312 (+ 4049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.184797 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.185608 (+ 811us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.185828 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.185944 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.186260 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6387,"thread_start_us":6241,"threads_started":1}
I20260812 02:48:11.188652 8170 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca454037f744b7b8d86ed929e05d711 (DEFAULT_TABLE table=test-table [id=4fe7e6c007ee4a0888f6b05a66b722fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:11.190599 8170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca454037f744b7b8d86ed929e05d711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:11.191067 8322 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca454037f744b7b8d86ed929e05d711 (DEFAULT_TABLE table=test-table [id=4fe7e6c007ee4a0888f6b05a66b722fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:11.192096 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.171274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.17.196.195:33705)
0812 02:48:11.182211 (+ 10937us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.182220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.182261 (+ 41us) server_negotiation.cc:407] Connection header received
0812 02:48:11.182381 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.182390 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.182713 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.183046 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:11.185157 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.186643 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.189167 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.190798 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.191014 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.191215 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.191513 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.191702 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10709,"thread_start_us":10510,"threads_started":1}
I20260812 02:48:11.192147 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.171099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:33705 (local address 127.0.0.1:45866)
0812 02:48:11.179780 (+ 8681us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:11.179803 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:11.179959 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:11.183369 (+ 3410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:11.183375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:11.183402 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:11.183858 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.183871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.186796 (+ 2925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.186803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.188878 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.188902 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.191064 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.191072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.191269 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.191308 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:48:11.191431 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 02:48:11.191603 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8502,"thread_start_us":8958,"threads_started":1}
I20260812 02:48:11.192641 8322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca454037f744b7b8d86ed929e05d711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:11.197340 8478 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca454037f744b7b8d86ed929e05d711 (DEFAULT_TABLE table=test-table [id=4fe7e6c007ee4a0888f6b05a66b722fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:11.200299 8478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca454037f744b7b8d86ed929e05d711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:11.206550 8579 tablet_bootstrap.cc:492] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Bootstrap starting.
I20260812 02:48:11.209267 8580 tablet_bootstrap.cc:492] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Bootstrap starting.
I20260812 02:48:11.213400 8580 tablet_bootstrap.cc:654] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:11.213399 8579 tablet_bootstrap.cc:654] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:11.215099 8580 log.cc:824] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:11.215086 8579 log.cc:824] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:11.217042 8584 tablet_bootstrap.cc:492] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Bootstrap starting.
I20260812 02:48:11.219739 8580 tablet_bootstrap.cc:492] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: No bootstrap required, opened a new log
I20260812 02:48:11.220186 8579 tablet_bootstrap.cc:492] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: No bootstrap required, opened a new log
I20260812 02:48:11.220221 8580 ts_tablet_manager.cc:1397] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.003s
I20260812 02:48:11.220525 8579 ts_tablet_manager.cc:1397] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260812 02:48:11.221882 8584 tablet_bootstrap.cc:654] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:11.223129 8584 log.cc:824] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:11.226516 8584 tablet_bootstrap.cc:492] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: No bootstrap required, opened a new log
I20260812 02:48:11.226879 8584 ts_tablet_manager.cc:1397] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 02:48:11.233765 8580 raft_consensus.cc:348] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.233662 8579 raft_consensus.cc:348] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.234278 8579 raft_consensus.cc:374] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:11.234278 8580 raft_consensus.cc:374] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:11.234404 8580 raft_consensus.cc:729] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eeff1c07ad841128ef712056bb2d241, State: Initialized, Role: FOLLOWER
I20260812 02:48:11.234404 8579 raft_consensus.cc:729] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e15108b6f3984e28a7a580ef910dc06c, State: Initialized, Role: FOLLOWER
I20260812 02:48:11.235211 8580 consensus_queue.cc:260] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.235211 8579 consensus_queue.cc:260] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.237195 8401 heartbeater.cc:503] Master 127.17.196.254:37619 was elected leader, sending a full tablet report...
I20260812 02:48:11.237663 8579 ts_tablet_manager.cc:1428] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Time spent starting tablet: real 0.017s user 0.019s sys 0.000s
I20260812 02:48:11.237315 8584 raft_consensus.cc:348] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.237823 8580 ts_tablet_manager.cc:1428] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Time spent starting tablet: real 0.017s user 0.015s sys 0.003s
I20260812 02:48:11.237896 8584 raft_consensus.cc:374] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:11.238030 8584 raft_consensus.cc:729] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e12b49051d43463e8704f391c6fcc8af, State: Initialized, Role: FOLLOWER
I20260812 02:48:11.238883 8584 consensus_queue.cc:260] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.240235 8557 heartbeater.cc:503] Master 127.17.196.254:37619 was elected leader, sending a full tablet report...
I20260812 02:48:11.241087 8584 ts_tablet_manager.cc:1428] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260812 02:48:11.303858 8248 tablet.cc:2369] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:48:11.320484 8558 tablet.cc:2369] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:48:11.432037 8402 tablet.cc:2369] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:48:11.435571 8587 raft_consensus.cc:482] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:48:11.436055 8587 raft_consensus.cc:504] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.438643 8587 leader_election.cc:288] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e15108b6f3984e28a7a580ef910dc06c (127.17.196.193:37095), e12b49051d43463e8704f391c6fcc8af (127.17.196.195:33705)
I20260812 02:48:11.447012 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.439647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:55421 (local address 127.17.196.193:37095)
0812 02:48:11.439888 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.439895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.440433 (+ 538us) server_negotiation.cc:407] Connection header received
0812 02:48:11.440779 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.440784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.440855 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.440981 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:11.441728 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.442611 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.444845 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.445875 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.446111 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.446204 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.446539 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.446695 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 02:48:11.447180 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.439013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:37095 (local address 127.17.196.194:55421)
0812 02:48:11.440266 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:11.440297 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:11.440576 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:11.441006 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:11.441011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:11.441041 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:11.441468 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.441554 (+ 86us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.442789 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.442795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.444596 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.444616 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.446138 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.446146 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.446340 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.446371 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 02:48:11.446505 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 02:48:11.446682 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":1,"thread_start_us":735,"threads_started":1}
I20260812 02:48:11.447505 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.439257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:34187 (local address 127.17.196.195:33705)
0812 02:48:11.439516 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.439527 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.440712 (+ 1185us) server_negotiation.cc:407] Connection header received
0812 02:48:11.441063 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.441069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.441151 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.441222 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:11.442304 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.443132 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.445267 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.446310 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.446535 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.446632 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.447036 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.447213 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 02:48:11.448001 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.439218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:33705 (local address 127.17.196.194:34187)
0812 02:48:11.440575 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:11.440601 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:11.440856 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:11.441463 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:11.441468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:11.441524 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:11.442034 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.442051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.443309 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.443316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.445051 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:11.445067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.446597 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.446605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:11.446795 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.446823 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 02:48:11.446980 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 02:48:11.447507 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"mutex_wait_us":64,"thread_start_us":951,"threads_started":1}
I20260812 02:48:11.448408 8194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca454037f744b7b8d86ed929e05d711" candidate_uuid: "6eeff1c07ad841128ef712056bb2d241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e15108b6f3984e28a7a580ef910dc06c" is_pre_election: true
I20260812 02:48:11.449133 8194 raft_consensus.cc:2454] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eeff1c07ad841128ef712056bb2d241 in term 0.
I20260812 02:48:11.449205 8501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca454037f744b7b8d86ed929e05d711" candidate_uuid: "6eeff1c07ad841128ef712056bb2d241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e12b49051d43463e8704f391c6fcc8af" is_pre_election: true
I20260812 02:48:11.449913 8501 raft_consensus.cc:2454] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eeff1c07ad841128ef712056bb2d241 in term 0.
I20260812 02:48:11.450155 8272 leader_election.cc:302] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eeff1c07ad841128ef712056bb2d241, e15108b6f3984e28a7a580ef910dc06c; no voters:
I20260812 02:48:11.450700 8587 raft_consensus.cc:2781] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:48:11.450884 8587 raft_consensus.cc:482] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:48:11.451031 8587 raft_consensus.cc:3037] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:11.453902 8587 raft_consensus.cc:504] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.454718 8587 leader_election.cc:288] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [CANDIDATE]: Term 1 election: Requested vote from peers e15108b6f3984e28a7a580ef910dc06c (127.17.196.193:37095), e12b49051d43463e8704f391c6fcc8af (127.17.196.195:33705)
I20260812 02:48:11.455425 8501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca454037f744b7b8d86ed929e05d711" candidate_uuid: "6eeff1c07ad841128ef712056bb2d241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e12b49051d43463e8704f391c6fcc8af"
I20260812 02:48:11.455430 8194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca454037f744b7b8d86ed929e05d711" candidate_uuid: "6eeff1c07ad841128ef712056bb2d241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e15108b6f3984e28a7a580ef910dc06c"
I20260812 02:48:11.455694 8194 raft_consensus.cc:3037] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:11.455696 8501 raft_consensus.cc:3037] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:11.458333 8194 raft_consensus.cc:2454] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eeff1c07ad841128ef712056bb2d241 in term 1.
I20260812 02:48:11.458392 8501 raft_consensus.cc:2454] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eeff1c07ad841128ef712056bb2d241 in term 1.
I20260812 02:48:11.458971 8272 leader_election.cc:302] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eeff1c07ad841128ef712056bb2d241, e15108b6f3984e28a7a580ef910dc06c; no voters:
I20260812 02:48:11.459523 8587 raft_consensus.cc:2781] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:11.460327 8587 raft_consensus.cc:686] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 1 LEADER]: Becoming Leader. State: Replica: 6eeff1c07ad841128ef712056bb2d241, State: Running, Role: LEADER
I20260812 02:48:11.460912 8587 consensus_queue.cc:237] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } }
I20260812 02:48:11.467444 8016 catalog_manager.cc:5495] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6eeff1c07ad841128ef712056bb2d241 (127.17.196.194). New cstate: current_term: 1 leader_uuid: "6eeff1c07ad841128ef712056bb2d241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeff1c07ad841128ef712056bb2d241" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 40429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 } health_report { overall_health: UNKNOWN } } }
I20260812 02:48:11.558841 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.549901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.17.196.194:40429)
0812 02:48:11.550175 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.550184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.550234 (+ 50us) server_negotiation.cc:407] Connection header received
0812 02:48:11.550562 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.550571 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.550846 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.551238 (+ 392us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:48:11.551837 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.552921 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.555440 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.555972 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.556278 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.557306 (+ 1028us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:48:11.558050 (+ 744us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:48:11.558202 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.558447 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.558545 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260812 02:48:11.571151 8194 raft_consensus.cc:1261] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 1 FOLLOWER]: Refusing update from remote peer 6eeff1c07ad841128ef712056bb2d241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:48:11.571146 8501 raft_consensus.cc:1261] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 1 FOLLOWER]: Refusing update from remote peer 6eeff1c07ad841128ef712056bb2d241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:48:11.572783 8587 consensus_queue.cc:1035] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e12b49051d43463e8704f391c6fcc8af" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:48:11.573261 8591 consensus_queue.cc:1035] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e15108b6f3984e28a7a580ef910dc06c" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 37095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:48:11.587211 8594 mvcc.cc:205] Tried to move back new op lower bound from 7317515843859234816 to 7317515843429244928. Current Snapshot: MvccSnapshot[applied={T|T < 7317515843859234816}]
I20260812 02:48:11.593688 8596 mvcc.cc:205] Tried to move back new op lower bound from 7317515843859234816 to 7317515843429244928. Current Snapshot: MvccSnapshot[applied={T|T < 7317515843859234816}]
I20260812 02:48:11.598632 8595 mvcc.cc:205] Tried to move back new op lower bound from 7317515843859234816 to 7317515843429244928. Current Snapshot: MvccSnapshot[applied={T|T < 7317515843859234816}]
I20260812 02:48:11.607161 8016 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:48:11.607652 8016 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:48:11.612847 8016 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efa3116999a449c4b6be834e853c019e: Sending DeleteTablet for 3 replicas of tablet 6ca454037f744b7b8d86ed929e05d711
I20260812 02:48:11.614705 8322 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca454037f744b7b8d86ed929e05d711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:35950
I20260812 02:48:11.614711 8170 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca454037f744b7b8d86ed929e05d711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:39196
I20260812 02:48:11.616258 8478 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca454037f744b7b8d86ed929e05d711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:11 UTC) from {username='kudu', principal='kudu/127.17.196.254@KRBTEST.COM'} at 127.0.0.1:45866
I20260812 02:48:11.617676 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 8094
I20260812 02:48:11.624279 8606 tablet_replica.cc:330] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: stopping tablet replica
I20260812 02:48:11.625010 8606 raft_consensus.cc:2229] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:48:11.625288 8605 tablet_replica.cc:330] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: stopping tablet replica
I20260812 02:48:11.625698 8606 raft_consensus.cc:2258] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:11.625926 8605 raft_consensus.cc:2229] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:48:11.626796 8605 raft_consensus.cc:2258] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:11.628654 8606 ts_tablet_manager.cc:1905] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:11.629663 8605 ts_tablet_manager.cc:1905] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:11.637775 8606 ts_tablet_manager.cc:1918] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:11.638151 8606 log.cc:1196] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-0/wal/wals/6ca454037f744b7b8d86ed929e05d711
I20260812 02:48:11.638826 8606 ts_tablet_manager.cc:1939] T 6ca454037f744b7b8d86ed929e05d711 P e15108b6f3984e28a7a580ef910dc06c: Deleting consensus metadata
I20260812 02:48:11.640887 8605 ts_tablet_manager.cc:1918] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:11.641088 8607 tablet_replica.cc:330] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: stopping tablet replica
I20260812 02:48:11.641757 8607 raft_consensus.cc:2229] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:48:11.641881 7999 catalog_manager.cc:4841] TS e15108b6f3984e28a7a580ef910dc06c (127.17.196.193:37095): tablet 6ca454037f744b7b8d86ed929e05d711 (table test-table [id=4fe7e6c007ee4a0888f6b05a66b722fd]) successfully deleted
I20260812 02:48:11.642165 8605 log.cc:1196] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/6ca454037f744b7b8d86ed929e05d711
I20260812 02:48:11.642432 8607 raft_consensus.cc:2258] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:11.644021 8605 ts_tablet_manager.cc:1939] T 6ca454037f744b7b8d86ed929e05d711 P 6eeff1c07ad841128ef712056bb2d241: Deleting consensus metadata
Aug 12 02:48:11 dist-test-slave-ss4n krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.193@KRBTEST.COM
I20260812 02:48:11.645783 8607 ts_tablet_manager.cc:1905] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:11.647037 8001 catalog_manager.cc:4841] TS 6eeff1c07ad841128ef712056bb2d241 (127.17.196.194:40429): tablet 6ca454037f744b7b8d86ed929e05d711 (table test-table [id=4fe7e6c007ee4a0888f6b05a66b722fd]) successfully deleted
I20260812 02:48:11.654073 8607 ts_tablet_manager.cc:1918] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:11.654498 8607 log.cc:1196] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/6ca454037f744b7b8d86ed929e05d711
I20260812 02:48:11.655411 8607 ts_tablet_manager.cc:1939] T 6ca454037f744b7b8d86ed929e05d711 P e12b49051d43463e8704f391c6fcc8af: Deleting consensus metadata
I20260812 02:48:11.656505 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.619339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39208 (local address 127.17.196.193:37095)
0812 02:48:11.619593 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.619602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.627397 (+ 7795us) server_negotiation.cc:407] Connection header received
0812 02:48:11.627484 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.627489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.627791 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.627933 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:11.630598 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.631945 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.636092 (+ 4147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.637183 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.637331 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.646469 (+ 9138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:11.646496 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:11.646521 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:11.646586 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:11.651775 (+ 5189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.653058 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.653073 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.653078 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.653324 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.653995 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.654002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.654006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.654613 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:11.655187 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.655680 (+ 493us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.656006 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:48:11.657382 8241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:11.658576 8001 catalog_manager.cc:4841] TS e12b49051d43463e8704f391c6fcc8af (127.17.196.195:33705): tablet 6ca454037f744b7b8d86ed929e05d711 (table test-table [id=4fe7e6c007ee4a0888f6b05a66b722fd]) successfully deleted
I20260812 02:48:11.759212 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 8094
I20260812 02:48:11.791098 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 8254
Aug 12 02:48:11 dist-test-slave-ss4n krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.194@KRBTEST.COM
I20260812 02:48:11.810603 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.792035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35970 (local address 127.17.196.194:40429)
0812 02:48:11.792253 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.792260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.792336 (+ 76us) server_negotiation.cc:407] Connection header received
0812 02:48:11.792576 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.792583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.792692 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.792847 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:11.794544 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.795487 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.796967 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.797537 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.797722 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.803040 (+ 5318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:11.803094 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:11.803113 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:11.803160 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:11.806982 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.807945 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.807954 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.807957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.808144 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.808635 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.808644 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.808653 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.809054 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:11.809549 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.809885 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.810175 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 02:48:11.811424 8395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:11.936374 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 8254
I20260812 02:48:11.962224 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 8407
Aug 12 02:48:11 dist-test-slave-ss4n krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.195@KRBTEST.COM
I20260812 02:48:11.981917 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:11.963196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.17.196.195:33705)
0812 02:48:11.963823 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:11.963829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:11.963848 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:48:11.963908 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:11.963912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:11.963982 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:11.964092 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:11.965692 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.966629 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.968196 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:11.968656 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:11.968773 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:11.974082 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:11.974100 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:11.974119 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:11.974184 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:11.977785 (+ 3601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.978940 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.978950 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.978953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.979158 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:11.979789 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:11.979794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:11.979796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:11.980286 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:11.980632 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:11.981100 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 02:48:11.981321 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":325,"threads_started":1}
I20260812 02:48:11.982856 8552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:12.087790 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 8407
I20260812 02:48:12.112757 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 7979
I20260812 02:48:12.114281 8054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:12.233862 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 7979
2026-08-12T02:48:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5504 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T02:48:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:12Z Disabled control of system clock
I20260812 02:48:12.343456 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:38307
--webserver_interface=127.17.196.254
--webserver_port=0
--builtin_ntp_servers=127.17.196.212:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:38307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:48:12.737586 8619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:12.738008 8619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:12.738162 8619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:12.746157 8619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:12.746346 8619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:12.746447 8619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:12.746531 8619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:12.746608 8619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:12.757858 8619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:38307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:38307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:12.759577 8619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:12.761876 8619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:12.776515 8625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:12.776561 8628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:12.776695 8626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:12.778445 8619 server_base.cc:1034] running on GCE node
I20260812 02:48:12.779568 8619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:12.781306 8619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:12.782580 8619 hybrid_clock.cc:648] HybridClock initialized: now 1786502892782497 us; error 87 us; skew 500 ppm
I20260812 02:48:12.782961 8619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:48:12.786062 8619 webserver.cc:460] Webserver started at http://127.17.196.254:41769/ using document root <none> and password file <none>
I20260812 02:48:12.786887 8619 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:12.786993 8619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:12.787288 8619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:12.790679 8619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "40c9cf83d04b451690523a9cb4591fdc"
format_stamp: "Formatted at 2026-08-12 02:48:12 on dist-test-slave-ss4n"
server_key: "2f88ce2e092d3dc9dc283e152beafc55"
server_key_iv: "a694df36b7879ac9fbf854c52e10795f"
server_key_version: "encryptionkey@0"
I20260812 02:48:12.791635 8619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "40c9cf83d04b451690523a9cb4591fdc"
format_stamp: "Formatted at 2026-08-12 02:48:12 on dist-test-slave-ss4n"
server_key: "2f88ce2e092d3dc9dc283e152beafc55"
server_key_iv: "a694df36b7879ac9fbf854c52e10795f"
server_key_version: "encryptionkey@0"
I20260812 02:48:12.797919 8619 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:48:12.802884 8634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:12.804996 8619 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:48:12.805303 8619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "40c9cf83d04b451690523a9cb4591fdc"
format_stamp: "Formatted at 2026-08-12 02:48:12 on dist-test-slave-ss4n"
server_key: "2f88ce2e092d3dc9dc283e152beafc55"
server_key_iv: "a694df36b7879ac9fbf854c52e10795f"
server_key_version: "encryptionkey@0"
I20260812 02:48:12.805716 8619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:12.827704 8619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:12.828883 8619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:12.829293 8619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:12.854580 8619 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:38307
I20260812 02:48:12.854625 8696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:38307 every 8 connection(s)
I20260812 02:48:12.856315 8619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:12.860882 8697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:12.862524 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 8619
I20260812 02:48:12.862756 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:12.863382 18195 external_mini_cluster.cc:1428] Setting key 05a2e404230717e3f602143f01c0d67f
I20260812 02:48:12.877602 8697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc: Bootstrap starting.
I20260812 02:48:12.880939 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:12.865890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34920 (local address 127.17.196.254:38307)
0812 02:48:12.867296 (+ 1406us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:12.867315 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:12.867383 (+ 68us) server_negotiation.cc:407] Connection header received
0812 02:48:12.869327 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:12.869370 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:12.870175 (+ 805us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:12.870709 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:12.871383 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:12.873358 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:12.875012 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:12.876102 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:12.876309 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:12.877632 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:12.877646 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:12.877668 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:12.878462 (+ 794us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:12.878583 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:12.878623 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:12.878757 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:12.879065 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 02:48:12.879434 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":451,"threads_started":1}
I20260812 02:48:12.882130 8697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:12.883533 8697 log.cc:824] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:12.887709 8697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc: No bootstrap required, opened a new log
I20260812 02:48:12.896106 8697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c9cf83d04b451690523a9cb4591fdc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 38307 } }
I20260812 02:48:12.896600 8697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:12.896726 8697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c9cf83d04b451690523a9cb4591fdc, State: Initialized, Role: FOLLOWER
I20260812 02:48:12.897368 8697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c9cf83d04b451690523a9cb4591fdc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 38307 } }
I20260812 02:48:12.897708 8697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:12.897873 8697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:12.898065 8697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:12.900449 8697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c9cf83d04b451690523a9cb4591fdc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 38307 } }
I20260812 02:48:12.901079 8697 leader_election.cc:302] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c9cf83d04b451690523a9cb4591fdc; no voters:
I20260812 02:48:12.902001 8697 leader_election.cc:288] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:12.902104 8702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:12.904078 8702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [term 1 LEADER]: Becoming Leader. State: Replica: 40c9cf83d04b451690523a9cb4591fdc, State: Running, Role: LEADER
I20260812 02:48:12.904834 8702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c9cf83d04b451690523a9cb4591fdc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 38307 } }
I20260812 02:48:12.905838 8697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:12.908505 8705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c9cf83d04b451690523a9cb4591fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c9cf83d04b451690523a9cb4591fdc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 38307 } } }
I20260812 02:48:12.908767 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c9cf83d04b451690523a9cb4591fdc. Latest consensus state: current_term: 1 leader_uuid: "40c9cf83d04b451690523a9cb4591fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c9cf83d04b451690523a9cb4591fdc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 38307 } } }
I20260812 02:48:12.909687 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:12.909699 8705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:12.911968 8710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:12.918854 8710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:12.930400 8710 catalog_manager.cc:1347] Generated new cluster ID: 3b7bf228d123444588ee55d9f94a5a02
I20260812 02:48:12.930549 8710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:12.956601 8710 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:12.958527 8710 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:12.967732 8710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc: Generated new TSK 0
I20260812 02:48:12.968991 8710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:12.981679 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.193:0
--local_ip_for_outbound_sockets=127.17.196.193
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:38307
--builtin_ntp_servers=127.17.196.212:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:48:13.404740 8726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:13.405068 8726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:13.405215 8726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:13.413739 8726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:13.413863 8726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:13.414081 8726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.193
I20260812 02:48:13.424225 8726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.196.193
--webserver_port=0
--tserver_master_addrs=127.17.196.254:38307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.193
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:13.425742 8726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:13.427713 8726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:13.442783 8732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:13.442845 8731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:13.444826 8734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:13.445166 8726 server_base.cc:1034] running on GCE node
I20260812 02:48:13.446092 8726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:13.447345 8726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:13.448612 8726 hybrid_clock.cc:648] HybridClock initialized: now 1786502893448553 us; error 48 us; skew 500 ppm
I20260812 02:48:13.449074 8726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:48:13.452236 8726 webserver.cc:460] Webserver started at http://127.17.196.193:44837/ using document root <none> and password file <none>
I20260812 02:48:13.453118 8726 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:13.453231 8726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:13.453735 8726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:13.457168 8726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/data/instance:
uuid: "39e108a8f9314776a28f760c3dcd89f5"
format_stamp: "Formatted at 2026-08-12 02:48:13 on dist-test-slave-ss4n"
server_key: "137723ee92502fff267de6f60151bda8"
server_key_iv: "c94d4908513c2b2606d4ffa74f564878"
server_key_version: "encryptionkey@0"
I20260812 02:48:13.458235 8726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance:
uuid: "39e108a8f9314776a28f760c3dcd89f5"
format_stamp: "Formatted at 2026-08-12 02:48:13 on dist-test-slave-ss4n"
server_key: "137723ee92502fff267de6f60151bda8"
server_key_iv: "c94d4908513c2b2606d4ffa74f564878"
server_key_version: "encryptionkey@0"
I20260812 02:48:13.464593 8726 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 02:48:13.469429 8743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:13.471642 8726 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 02:48:13.471891 8726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/wal
uuid: "39e108a8f9314776a28f760c3dcd89f5"
format_stamp: "Formatted at 2026-08-12 02:48:13 on dist-test-slave-ss4n"
server_key: "137723ee92502fff267de6f60151bda8"
server_key_iv: "c94d4908513c2b2606d4ffa74f564878"
server_key_version: "encryptionkey@0"
I20260812 02:48:13.472183 8726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:13.506534 8726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:13.507644 8726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:13.508020 8726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:13.509248 8726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:13.511379 8726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:13.511498 8726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:13.511656 8726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:13.511749 8726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:13.553329 8726 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.193:39811
I20260812 02:48:13.553539 8875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.193:39811 every 8 connection(s)
I20260812 02:48:13.554939 8726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/data/info.pb
I20260812 02:48:13.557554 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 8726
I20260812 02:48:13.557775 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-0/wal/instance
I20260812 02:48:13.558300 18195 external_mini_cluster.cc:1428] Setting key 395d09c4b87a05d50c57ccdc2b7b9782
I20260812 02:48:13.562528 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.194:0
--local_ip_for_outbound_sockets=127.17.196.194
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:38307
--builtin_ntp_servers=127.17.196.212:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:48:13.575982 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:13.558763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.193:54181 (local address 127.17.196.254:38307)
0812 02:48:13.559391 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:13.559399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:13.561415 (+ 2016us) server_negotiation.cc:407] Connection header received
0812 02:48:13.565341 (+ 3926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:13.565347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:13.565593 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:13.565714 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:13.567635 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:13.568455 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:13.571709 (+ 3254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:13.572230 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:13.572355 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:13.574752 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:13.574762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:13.574769 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:13.574802 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:13.574863 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:13.574883 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:13.574997 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:13.575416 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 02:48:13.575573 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":313,"threads_started":1}
I20260812 02:48:13.577767 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:13.559556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:38307 (local address 127.17.196.193:54181)
0812 02:48:13.561186 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:13.561260 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:13.564805 (+ 3545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:13.566080 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:13.566097 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:13.566269 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:13.567313 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:13.567347 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:13.568669 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:13.568679 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:13.571459 (+ 2780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:13.571481 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:13.572519 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:13.572530 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:13.572903 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:13.574255 (+ 1352us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:48:13.574262 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:13.574340 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:48:13.574473 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:13.575041 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:13.575056 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:13.575068 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 02:48:13.575317 (+ 249us) client_negotiation.cc:240] Negotiation successful
0812 02:48:13.575831 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":442,"threads_started":1}
I20260812 02:48:13.580247 8876 heartbeater.cc:347] Connected to a master server at 127.17.196.254:38307
I20260812 02:48:13.580657 8876 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:13.581712 8876 heartbeater.cc:511] Master 127.17.196.254:38307 requested a full tablet report, sending...
I20260812 02:48:13.584460 8654 ts_manager.cc:194] Registered new tserver with Master: 39e108a8f9314776a28f760c3dcd89f5 (127.17.196.193:39811)
I20260812 02:48:13.586853 8654 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.193:54181
W20260812 02:48:13.964798 8881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:13.965157 8881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:13.965257 8881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:13.973651 8881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:13.973786 8881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:13.973908 8881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.194
I20260812 02:48:13.984141 8881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.196.194
--webserver_port=0
--tserver_master_addrs=127.17.196.254:38307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.194
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:13.985710 8881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:13.987699 8881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:14.004063 8889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:14.003966 8891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:14.003957 8888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:14.004957 8881 server_base.cc:1034] running on GCE node
I20260812 02:48:14.005788 8881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:14.007063 8881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:14.008378 8881 hybrid_clock.cc:648] HybridClock initialized: now 1786502894008342 us; error 80 us; skew 500 ppm
I20260812 02:48:14.008917 8881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:48:14.017732 8881 webserver.cc:460] Webserver started at http://127.17.196.194:41715/ using document root <none> and password file <none>
I20260812 02:48:14.018672 8881 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:14.018862 8881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:14.019263 8881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:14.023048 8881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/data/instance:
uuid: "04923f6c2faf4d5cba8e625a10ce040d"
format_stamp: "Formatted at 2026-08-12 02:48:14 on dist-test-slave-ss4n"
server_key: "e10c4efb070172e437a4ef4b83c4cc5b"
server_key_iv: "0dfef37a22f07924541e34796e7b5fac"
server_key_version: "encryptionkey@0"
I20260812 02:48:14.024255 8881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance:
uuid: "04923f6c2faf4d5cba8e625a10ce040d"
format_stamp: "Formatted at 2026-08-12 02:48:14 on dist-test-slave-ss4n"
server_key: "e10c4efb070172e437a4ef4b83c4cc5b"
server_key_iv: "0dfef37a22f07924541e34796e7b5fac"
server_key_version: "encryptionkey@0"
I20260812 02:48:14.030961 8881 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 02:48:14.035660 8898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:14.037825 8881 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 02:48:14.038100 8881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/wal
uuid: "04923f6c2faf4d5cba8e625a10ce040d"
format_stamp: "Formatted at 2026-08-12 02:48:14 on dist-test-slave-ss4n"
server_key: "e10c4efb070172e437a4ef4b83c4cc5b"
server_key_iv: "0dfef37a22f07924541e34796e7b5fac"
server_key_version: "encryptionkey@0"
I20260812 02:48:14.038391 8881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:14.055435 8881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:14.056463 8881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:14.056852 8881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:14.058182 8881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:14.060060 8881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:14.060209 8881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:14.060370 8881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:14.060472 8881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:14.102018 8881 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.194:39313
I20260812 02:48:14.102056 9038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.194:39313 every 8 connection(s)
I20260812 02:48:14.103544 8881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/data/info.pb
I20260812 02:48:14.106458 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 8881
I20260812 02:48:14.106689 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/wal/instance
I20260812 02:48:14.107342 18195 external_mini_cluster.cc:1428] Setting key cb2664d12d2b58ce1d8ec561a9eee671
I20260812 02:48:14.111275 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.196.195:0
--local_ip_for_outbound_sockets=127.17.196.195
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:38307
--builtin_ntp_servers=127.17.196.212:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:48:14.119947 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.106847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:57247 (local address 127.17.196.254:38307)
0812 02:48:14.107452 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:14.107460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:14.109283 (+ 1823us) server_negotiation.cc:407] Connection header received
0812 02:48:14.111844 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:14.111849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:14.111955 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:14.112092 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:14.113862 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.114694 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.116533 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.117026 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.117187 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.118909 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:14.118914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:14.118918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:14.118941 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:14.118988 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:14.119003 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:14.119114 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:14.119398 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 02:48:14.119521 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":291,"threads_started":1}
I20260812 02:48:14.121141 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.107727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:38307 (local address 127.17.196.194:57247)
0812 02:48:14.109067 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:14.109134 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:14.111423 (+ 2289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:14.112409 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:14.112419 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:14.112529 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:14.113620 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.113649 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.114889 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.114895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.116345 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.116361 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.117305 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.117313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.117661 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.118508 (+ 847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:48:14.118511 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:14.118560 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:48:14.118658 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:14.119119 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:14.119135 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:14.119146 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 02:48:14.119355 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 02:48:14.119848 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":428,"threads_started":1}
I20260812 02:48:14.122936 9039 heartbeater.cc:347] Connected to a master server at 127.17.196.254:38307
I20260812 02:48:14.123308 9039 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:14.124184 9039 heartbeater.cc:511] Master 127.17.196.254:38307 requested a full tablet report, sending...
I20260812 02:48:14.126041 8654 ts_manager.cc:194] Registered new tserver with Master: 04923f6c2faf4d5cba8e625a10ce040d (127.17.196.194:39313)
I20260812 02:48:14.127075 8654 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.194:57247
W20260812 02:48:14.510303 9043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:14.510658 9043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:14.510810 9043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:14.519161 9043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:14.519289 9043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:14.519573 9043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.196.195
I20260812 02:48:14.529762 9043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.196.195
--webserver_port=0
--tserver_master_addrs=127.17.196.254:38307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.196.195
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:14.531291 9043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:14.534035 9043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:14.552321 9050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:14.553843 9052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:14.553862 9049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:14.558184 9043 server_base.cc:1034] running on GCE node
I20260812 02:48:14.559082 9043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:14.560475 9043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:14.561810 9043 hybrid_clock.cc:648] HybridClock initialized: now 1786502894561722 us; error 78 us; skew 500 ppm
I20260812 02:48:14.562367 9043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:48:14.565615 9043 webserver.cc:460] Webserver started at http://127.17.196.195:32787/ using document root <none> and password file <none>
I20260812 02:48:14.566573 9043 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:14.567024 9043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:14.567546 9043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:14.571574 9043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/data/instance:
uuid: "124d75124cb546e48f3a7fb3abf1dd44"
format_stamp: "Formatted at 2026-08-12 02:48:14 on dist-test-slave-ss4n"
server_key: "71779db88fbdacb2c35c304605a6c09f"
server_key_iv: "654d4994f7fd146d1e9cb2c98ac3ef40"
server_key_version: "encryptionkey@0"
I20260812 02:48:14.572701 9043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance:
uuid: "124d75124cb546e48f3a7fb3abf1dd44"
format_stamp: "Formatted at 2026-08-12 02:48:14 on dist-test-slave-ss4n"
server_key: "71779db88fbdacb2c35c304605a6c09f"
server_key_iv: "654d4994f7fd146d1e9cb2c98ac3ef40"
server_key_version: "encryptionkey@0"
I20260812 02:48:14.580762 9043 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260812 02:48:14.586300 9060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:14.588203 9043 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:48:14.588462 9043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/wal
uuid: "124d75124cb546e48f3a7fb3abf1dd44"
format_stamp: "Formatted at 2026-08-12 02:48:14 on dist-test-slave-ss4n"
server_key: "71779db88fbdacb2c35c304605a6c09f"
server_key_iv: "654d4994f7fd146d1e9cb2c98ac3ef40"
server_key_version: "encryptionkey@0"
I20260812 02:48:14.588749 9043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:14.590909 8876 heartbeater.cc:503] Master 127.17.196.254:38307 was elected leader, sending a full tablet report...
I20260812 02:48:14.621924 9043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:14.622991 9043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:14.623330 9043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:14.624459 9043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:48:14.626711 9043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:48:14.626837 9043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:14.626929 9043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:48:14.627066 9043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:14.671610 9043 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.195:40095
I20260812 02:48:14.671731 9198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.195:40095 every 8 connection(s)
I20260812 02:48:14.673082 9043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/data/info.pb
I20260812 02:48:14.681205 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 9043
I20260812 02:48:14.681406 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/wal/instance
I20260812 02:48:14.682071 18195 external_mini_cluster.cc:1428] Setting key 5b5db792a5978698e9761a6c2f8ceab5
I20260812 02:48:14.690052 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.676350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.195:49845 (local address 127.17.196.254:38307)
0812 02:48:14.676862 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:14.676869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:14.678646 (+ 1777us) server_negotiation.cc:407] Connection header received
0812 02:48:14.680665 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:14.680673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:14.680760 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:14.680881 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:14.682771 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.683909 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.686450 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.686960 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.687142 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.688904 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:14.688911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:14.688931 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:14.688962 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:14.689030 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:14.689049 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:14.689131 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:14.689624 (+ 493us) server_negotiation.cc:299] Negotiation successful
0812 02:48:14.689737 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":218,"threads_started":1}
I20260812 02:48:14.691299 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.677136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:38307 (local address 127.17.196.195:49845)
0812 02:48:14.678435 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:14.678476 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:14.680311 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:14.681213 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:14.681228 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:14.681396 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:48:14.682497 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.682526 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.684118 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.684125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.686279 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.686297 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.687347 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.687354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.687613 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.688432 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:48:14.688435 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:48:14.688498 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:48:14.688615 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:48:14.689232 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:48:14.689243 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:48:14.689252 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 02:48:14.689449 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 02:48:14.689932 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":497,"threads_started":1}
I20260812 02:48:14.693132 9199 heartbeater.cc:347] Connected to a master server at 127.17.196.254:38307
I20260812 02:48:14.693605 9199 heartbeater.cc:464] Registering TS with master...
I20260812 02:48:14.694614 9199 heartbeater.cc:511] Master 127.17.196.254:38307 requested a full tablet report, sending...
I20260812 02:48:14.696410 8654 ts_manager.cc:194] Registered new tserver with Master: 124d75124cb546e48f3a7fb3abf1dd44 (127.17.196.195:40095)
I20260812 02:48:14.697633 8654 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.196.195:49845
I20260812 02:48:14.699093 18195 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:48:14.711701 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.703980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.17.196.254:38307)
0812 02:48:14.704186 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:14.704191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:14.704592 (+ 401us) server_negotiation.cc:407] Connection header received
0812 02:48:14.704758 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:14.704762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:14.704833 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:14.704939 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:14.705730 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.706653 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.708450 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.708970 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.709096 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.710669 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:14.710675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:14.710686 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:14.710715 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:14.710759 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:14.710777 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:14.710875 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:14.711215 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 02:48:14.711318 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:48:14.718037 8654 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:48:14.727154 8654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:48:14.767159 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.744458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40095 (local address 127.0.0.1:53034)
0812 02:48:14.755206 (+ 10748us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:14.755289 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:14.755756 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:14.757744 (+ 1988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:14.757753 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:14.757793 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:14.758483 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.758508 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.761202 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.761208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.764004 (+ 2796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.764023 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.766073 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.766081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.766303 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.766360 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 02:48:14.766535 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 02:48:14.766790 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10238,"thread_start_us":10010,"threads_started":1}
I20260812 02:48:14.767213 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.744666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.17.196.195:40095)
0812 02:48:14.756334 (+ 11668us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:14.756345 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:14.756394 (+ 49us) server_negotiation.cc:407] Connection header received
0812 02:48:14.756534 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:14.756546 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:14.756955 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:14.757283 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:14.758985 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.760977 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.764292 (+ 3315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.765835 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.766073 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.766355 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:14.766605 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 02:48:14.766799 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11461,"thread_start_us":11276,"threads_started":1}
I20260812 02:48:14.768536 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.747927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:42978)
0812 02:48:14.759055 (+ 11128us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:14.759085 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:14.759363 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:14.760636 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:14.760640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:14.760663 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:14.760976 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.760983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.762717 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.762727 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.765029 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.765047 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.767643 (+ 2596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.767649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.767813 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.767848 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:48:14.767934 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 02:48:14.768132 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10981,"mutex_wait_us":1,"spinlock_wait_cycles":4864,"thread_start_us":9233,"threads_started":1}
I20260812 02:48:14.769625 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.748840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42978 (local address 127.17.196.193:39811)
0812 02:48:14.756947 (+ 8107us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:14.756957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:14.759218 (+ 2261us) server_negotiation.cc:407] Connection header received
0812 02:48:14.759662 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:14.759673 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:14.760083 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:14.760297 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:14.761164 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.762540 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.765275 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.766958 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.767189 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.768954 (+ 1765us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:14.769055 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 02:48:14.769249 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7829,"thread_start_us":7682,"threads_started":1}
I20260812 02:48:14.769714 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.744745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.194:39313 (local address 127.0.0.1:48422)
0812 02:48:14.757595 (+ 12850us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:14.757629 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:14.757766 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:14.759269 (+ 1503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:14.759274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:14.759336 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:14.759835 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.759850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.762348 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.762354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.764671 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.764684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.767280 (+ 2596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.767287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.767454 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.767487 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:48:14.769165 (+ 1678us) client_negotiation.cc:240] Negotiation successful
0812 02:48:14.769308 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12628,"mutex_wait_us":8,"spinlock_wait_cycles":4352,"thread_start_us":13917,"threads_started":1}
I20260812 02:48:14.770195 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.744936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.17.196.194:39313)
0812 02:48:14.758380 (+ 13444us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:14.758406 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:14.758459 (+ 53us) server_negotiation.cc:407] Connection header received
0812 02:48:14.758590 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:14.758602 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:14.758983 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:14.760082 (+ 1099us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:14.760336 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.762164 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.764900 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.766691 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.766880 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.767121 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:14.769246 (+ 2125us) server_negotiation.cc:299] Negotiation successful
0812 02:48:14.769451 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13243,"thread_start_us":13117,"threads_started":1}
I20260812 02:48:14.772033 9120 tablet_service.cc:1467] Processing CreateTablet for tablet a69833e7daad4080a61061db40e0befe (DEFAULT_TABLE table=test-table [id=acb663a00758451088c2cfe7f59cd707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:14.773785 9120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69833e7daad4080a61061db40e0befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:14.773599 8960 tablet_service.cc:1467] Processing CreateTablet for tablet a69833e7daad4080a61061db40e0befe (DEFAULT_TABLE table=test-table [id=acb663a00758451088c2cfe7f59cd707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:14.773607 8802 tablet_service.cc:1467] Processing CreateTablet for tablet a69833e7daad4080a61061db40e0befe (DEFAULT_TABLE table=test-table [id=acb663a00758451088c2cfe7f59cd707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:48:14.775034 8960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69833e7daad4080a61061db40e0befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:14.775085 8802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69833e7daad4080a61061db40e0befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:14.790742 9223 tablet_bootstrap.cc:492] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Bootstrap starting.
I20260812 02:48:14.795356 9223 tablet_bootstrap.cc:654] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:14.796486 9224 tablet_bootstrap.cc:492] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Bootstrap starting.
I20260812 02:48:14.796690 9223 log.cc:824] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:14.800307 9225 tablet_bootstrap.cc:492] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5: Bootstrap starting.
I20260812 02:48:14.804939 9225 tablet_bootstrap.cc:654] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:14.804941 9224 tablet_bootstrap.cc:654] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:14.806550 9225 log.cc:824] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:14.806536 9224 log.cc:824] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:14.808959 9223 tablet_bootstrap.cc:492] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: No bootstrap required, opened a new log
I20260812 02:48:14.809346 9223 ts_tablet_manager.cc:1397] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260812 02:48:14.810220 9225 tablet_bootstrap.cc:492] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5: No bootstrap required, opened a new log
I20260812 02:48:14.810550 9225 ts_tablet_manager.cc:1397] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260812 02:48:14.812007 9224 tablet_bootstrap.cc:492] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: No bootstrap required, opened a new log
I20260812 02:48:14.812330 9224 ts_tablet_manager.cc:1397] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.008s
I20260812 02:48:14.818717 9223 raft_consensus.cc:348] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.819242 9223 raft_consensus.cc:374] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:14.819363 9223 raft_consensus.cc:729] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04923f6c2faf4d5cba8e625a10ce040d, State: Initialized, Role: FOLLOWER
I20260812 02:48:14.820138 9223 consensus_queue.cc:260] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.821789 9039 heartbeater.cc:503] Master 127.17.196.254:38307 was elected leader, sending a full tablet report...
I20260812 02:48:14.822842 9223 ts_tablet_manager.cc:1428] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260812 02:48:14.822685 9225 raft_consensus.cc:348] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.823212 9225 raft_consensus.cc:374] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:14.823319 9225 raft_consensus.cc:729] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e108a8f9314776a28f760c3dcd89f5, State: Initialized, Role: FOLLOWER
I20260812 02:48:14.824043 9225 consensus_queue.cc:260] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.823877 9224 raft_consensus.cc:348] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.824420 9224 raft_consensus.cc:374] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:14.824541 9224 raft_consensus.cc:729] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 124d75124cb546e48f3a7fb3abf1dd44, State: Initialized, Role: FOLLOWER
I20260812 02:48:14.825601 9224 consensus_queue.cc:260] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.827512 9225 ts_tablet_manager.cc:1428] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
I20260812 02:48:14.828984 9199 heartbeater.cc:503] Master 127.17.196.254:38307 was elected leader, sending a full tablet report...
I20260812 02:48:14.831543 9224 ts_tablet_manager.cc:1428] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
W20260812 02:48:14.856312 9040 tablet.cc:2369] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:48:14.925381 9200 tablet.cc:2369] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:48:14.940299 9231 raft_consensus.cc:482] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:48:14.940791 9231 raft_consensus.cc:504] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.943177 9231 leader_election.cc:288] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 124d75124cb546e48f3a7fb3abf1dd44 (127.17.196.195:40095), 39e108a8f9314776a28f760c3dcd89f5 (127.17.196.193:39811)
I20260812 02:48:14.952452 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.943918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:53929 (local address 127.17.196.193:39811)
0812 02:48:14.944263 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:14.944274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:14.945244 (+ 970us) server_negotiation.cc:407] Connection header received
0812 02:48:14.945563 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:14.945568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:14.945663 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:14.945796 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:14.947027 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.947843 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.950257 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.951269 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.951589 (+ 320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.951699 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:14.951992 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 02:48:14.952152 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260812 02:48:14.952999 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.943521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.17.196.194:53929)
0812 02:48:14.945062 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:14.945088 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:14.945333 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:14.945820 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:14.945825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:14.945852 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:14.946797 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.946813 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.948030 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.948036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.949982 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.950004 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.951554 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.951566 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.951781 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.951810 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:48:14.951949 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 02:48:14.952142 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1346,"mutex_wait_us":3,"thread_start_us":1060,"threads_started":1}
I20260812 02:48:14.953116 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.943791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.196.194:38149 (local address 127.17.196.195:40095)
0812 02:48:14.944081 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:14.944091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:14.945305 (+ 1214us) server_negotiation.cc:407] Connection header received
0812 02:48:14.945564 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:14.945570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:14.945662 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:14.945785 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:48:14.946817 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.947633 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.950245 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.951519 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.951836 (+ 317us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.951909 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:14.952263 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 02:48:14.952401 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 02:48:14.953214 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:14.943872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.195:40095 (local address 127.17.196.194:38149)
0812 02:48:14.945148 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:14.945171 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:48:14.945366 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:48:14.945808 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:48:14.945813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:48:14.945840 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:48:14.946518 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.946541 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.947808 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.947816 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.949926 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:48:14.949948 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:48:14.951842 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:48:14.951848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:48:14.952056 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:14.952102 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 02:48:14.952227 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 02:48:14.952372 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1159,"mutex_wait_us":76,"thread_start_us":919,"threads_started":1}
I20260812 02:48:14.954537 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69833e7daad4080a61061db40e0befe" candidate_uuid: "04923f6c2faf4d5cba8e625a10ce040d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124d75124cb546e48f3a7fb3abf1dd44" is_pre_election: true
I20260812 02:48:14.954416 8824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69833e7daad4080a61061db40e0befe" candidate_uuid: "04923f6c2faf4d5cba8e625a10ce040d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e108a8f9314776a28f760c3dcd89f5" is_pre_election: true
I20260812 02:48:14.955164 8824 raft_consensus.cc:2454] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04923f6c2faf4d5cba8e625a10ce040d in term 0.
I20260812 02:48:14.955144 9147 raft_consensus.cc:2454] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04923f6c2faf4d5cba8e625a10ce040d in term 0.
I20260812 02:48:14.956207 8900 leader_election.cc:302] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04923f6c2faf4d5cba8e625a10ce040d, 39e108a8f9314776a28f760c3dcd89f5; no voters:
I20260812 02:48:14.956804 9231 raft_consensus.cc:2781] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:48:14.957005 9231 raft_consensus.cc:482] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:48:14.957155 9231 raft_consensus.cc:3037] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:14.959770 9231 raft_consensus.cc:504] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.960512 9231 leader_election.cc:288] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [CANDIDATE]: Term 1 election: Requested vote from peers 124d75124cb546e48f3a7fb3abf1dd44 (127.17.196.195:40095), 39e108a8f9314776a28f760c3dcd89f5 (127.17.196.193:39811)
I20260812 02:48:14.961321 9147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69833e7daad4080a61061db40e0befe" candidate_uuid: "04923f6c2faf4d5cba8e625a10ce040d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124d75124cb546e48f3a7fb3abf1dd44"
I20260812 02:48:14.961369 8824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69833e7daad4080a61061db40e0befe" candidate_uuid: "04923f6c2faf4d5cba8e625a10ce040d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e108a8f9314776a28f760c3dcd89f5"
I20260812 02:48:14.961795 9147 raft_consensus.cc:3037] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:14.961824 8824 raft_consensus.cc:3037] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:14.964955 9147 raft_consensus.cc:2454] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04923f6c2faf4d5cba8e625a10ce040d in term 1.
I20260812 02:48:14.964956 8824 raft_consensus.cc:2454] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04923f6c2faf4d5cba8e625a10ce040d in term 1.
I20260812 02:48:14.965711 8900 leader_election.cc:302] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04923f6c2faf4d5cba8e625a10ce040d, 39e108a8f9314776a28f760c3dcd89f5; no voters:
I20260812 02:48:14.966197 9231 raft_consensus.cc:2781] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:14.966889 9231 raft_consensus.cc:686] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 1 LEADER]: Becoming Leader. State: Replica: 04923f6c2faf4d5cba8e625a10ce040d, State: Running, Role: LEADER
I20260812 02:48:14.967793 9231 consensus_queue.cc:237] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } }
I20260812 02:48:14.973719 8653 catalog_manager.cc:5495] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d reported cstate change: term changed from 0 to 1, leader changed from <none> to 04923f6c2faf4d5cba8e625a10ce040d (127.17.196.194). New cstate: current_term: 1 leader_uuid: "04923f6c2faf4d5cba8e625a10ce040d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04923f6c2faf4d5cba8e625a10ce040d" member_type: VOTER last_known_addr { host: "127.17.196.194" port: 39313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 } health_report { overall_health: UNKNOWN } } }
I20260812 02:48:15.020140 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.012345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.17.196.194:39313)
0812 02:48:15.012579 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:15.012584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:15.012711 (+ 127us) server_negotiation.cc:407] Connection header received
0812 02:48:15.012870 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:15.012874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:15.013135 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:15.013347 (+ 212us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:48:15.014130 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:15.014936 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:15.017049 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:15.017804 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:15.017961 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:15.018659 (+ 698us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:48:15.019267 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:48:15.019454 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:15.019570 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 02:48:15.019664 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:48:15.030376 8824 raft_consensus.cc:1261] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 1 FOLLOWER]: Refusing update from remote peer 04923f6c2faf4d5cba8e625a10ce040d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:48:15.030376 9147 raft_consensus.cc:1261] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 1 FOLLOWER]: Refusing update from remote peer 04923f6c2faf4d5cba8e625a10ce040d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:48:15.031623 9236 consensus_queue.cc:1035] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [LEADER]: Connected to new peer: Peer: permanent_uuid: "124d75124cb546e48f3a7fb3abf1dd44" member_type: VOTER last_known_addr { host: "127.17.196.195" port: 40095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:48:15.032119 9231 consensus_queue.cc:1035] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39e108a8f9314776a28f760c3dcd89f5" member_type: VOTER last_known_addr { host: "127.17.196.193" port: 39811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:48:15.047317 9239 mvcc.cc:205] Tried to move back new op lower bound from 7317515858031718400 to 7317515857793359872. Current Snapshot: MvccSnapshot[applied={T|T < 7317515858031718400}]
I20260812 02:48:15.048588 9241 mvcc.cc:205] Tried to move back new op lower bound from 7317515858031718400 to 7317515857793359872. Current Snapshot: MvccSnapshot[applied={T|T < 7317515858031718400}]
I20260812 02:48:15.068040 8654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:48:15.068373 8654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:48:15.073284 8654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40c9cf83d04b451690523a9cb4591fdc: Sending DeleteTablet for 3 replicas of tablet a69833e7daad4080a61061db40e0befe
I20260812 02:48:15.075142 8802 tablet_service.cc:1514] Processing DeleteTablet for tablet a69833e7daad4080a61061db40e0befe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:15 UTC) from {username='slave'} at 127.0.0.1:42978
I20260812 02:48:15.075127 9120 tablet_service.cc:1514] Processing DeleteTablet for tablet a69833e7daad4080a61061db40e0befe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:15 UTC) from {username='slave'} at 127.0.0.1:53034
I20260812 02:48:15.075136 8960 tablet_service.cc:1514] Processing DeleteTablet for tablet a69833e7daad4080a61061db40e0befe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:48:15 UTC) from {username='slave'} at 127.0.0.1:48422
I20260812 02:48:15.079993 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 8726
I20260812 02:48:15.088181 9251 tablet_replica.cc:330] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5: stopping tablet replica
I20260812 02:48:15.088768 9251 raft_consensus.cc:2229] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:48:15.089433 9251 raft_consensus.cc:2258] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:15.090785 9250 tablet_replica.cc:330] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: stopping tablet replica
I20260812 02:48:15.091472 9250 raft_consensus.cc:2229] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:48:15.092206 9250 raft_consensus.cc:2258] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:15.094234 9252 tablet_replica.cc:330] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: stopping tablet replica
I20260812 02:48:15.094961 9252 raft_consensus.cc:2229] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:48:15.095640 9251 ts_tablet_manager.cc:1905] T a69833e7daad4080a61061db40e0befe P 39e108a8f9314776a28f760c3dcd89f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:15.096117 9252 raft_consensus.cc:2258] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:48:15.099027 9250 ts_tablet_manager.cc:1905] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:15.099061 9252 ts_tablet_manager.cc:1905] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:48:15.100126 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.082020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.17.196.193:39811)
0812 02:48:15.082264 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:15.082272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:15.088513 (+ 6241us) server_negotiation.cc:407] Connection header received
0812 02:48:15.088926 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:15.088936 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:15.089106 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:15.089221 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:15.090160 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:15.091742 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:15.094792 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:15.095579 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:15.095727 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:15.097992 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:15.098005 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:15.098024 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:15.098956 (+ 932us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:15.099036 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:15.099072 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:15.099208 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:15.099518 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 02:48:15.099690 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 02:48:15.106814 8870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:15.108927 9250 ts_tablet_manager.cc:1918] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:15.109277 9250 log.cc:1196] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-2/wal/wals/a69833e7daad4080a61061db40e0befe
I20260812 02:48:15.110365 9250 ts_tablet_manager.cc:1939] T a69833e7daad4080a61061db40e0befe P 124d75124cb546e48f3a7fb3abf1dd44: Deleting consensus metadata
I20260812 02:48:15.113473 8638 catalog_manager.cc:4841] TS 124d75124cb546e48f3a7fb3abf1dd44 (127.17.196.195:40095): tablet a69833e7daad4080a61061db40e0befe (table test-table [id=acb663a00758451088c2cfe7f59cd707]) successfully deleted
I20260812 02:48:15.115706 9252 ts_tablet_manager.cc:1918] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:48:15.115950 9252 log.cc:1196] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Deleting WAL directory at /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502547576936-18195-0/minicluster-data/ts-1/wal/wals/a69833e7daad4080a61061db40e0befe
I20260812 02:48:15.116714 9252 ts_tablet_manager.cc:1939] T a69833e7daad4080a61061db40e0befe P 04923f6c2faf4d5cba8e625a10ce040d: Deleting consensus metadata
I20260812 02:48:15.119081 8640 catalog_manager.cc:4841] TS 04923f6c2faf4d5cba8e625a10ce040d (127.17.196.194:39313): tablet a69833e7daad4080a61061db40e0befe (table test-table [id=acb663a00758451088c2cfe7f59cd707]) successfully deleted
I20260812 02:48:15.243021 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 8726
I20260812 02:48:15.267477 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 8881
W20260812 02:48:15.267501 8636 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.196.193:39811 (error 108)
I20260812 02:48:15.269811 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.269206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:42992)
0812 02:48:15.269394 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:15.269610 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 02:48:15.270175 8636 catalog_manager.cc:4584] TS 39e108a8f9314776a28f760c3dcd89f5 (127.17.196.193:39811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a69833e7daad4080a61061db40e0befe: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
I20260812 02:48:15.277253 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.268793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.17.196.194:39313)
0812 02:48:15.269039 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:15.269049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:15.269080 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:48:15.269158 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:15.269164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:15.269306 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:15.269510 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:15.270049 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:15.271044 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:15.273261 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:15.274026 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:15.274150 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:15.275556 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:15.275565 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:15.275581 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:15.276267 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:15.276331 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:15.276353 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:15.276449 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:15.276680 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 02:48:15.276783 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 02:48:15.278090 9032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:15.324477 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.323767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:42998)
0812 02:48:15.324212 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:15.324313 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249}
I20260812 02:48:15.326154 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.325455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:43010)
0812 02:48:15.325756 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:15.325982 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260812 02:48:15.387557 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.386983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:43026)
0812 02:48:15.387213 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:15.387324 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 02:48:15.389294 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.388698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:43030)
0812 02:48:15.389019 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:15.389143 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260812 02:48:15.416576 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 8881
I20260812 02:48:15.445669 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9043
I20260812 02:48:15.455562 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.446761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53042 (local address 127.17.196.195:40095)
0812 02:48:15.446964 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:15.446969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:15.446992 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:48:15.447275 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:15.447280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:15.447374 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:15.447476 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:15.448173 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:15.449037 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:15.451408 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:15.451971 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:15.452106 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:15.453740 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:15.453752 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:15.453769 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:15.454523 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:15.454661 (+ 138us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:15.454691 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:15.454807 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:15.455120 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 02:48:15.455259 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:48:15.456313 9193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:15.470479 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.469911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:43042)
0812 02:48:15.470194 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:15.470289 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 02:48:15.471683 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.471245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:43048)
0812 02:48:15.471463 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:15.471554 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 02:48:15.569329 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 9043
I20260812 02:48:15.595109 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 8619
I20260812 02:48:15.596554 8692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:15.717270 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:15.716669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.193:39811 (local address 127.0.0.1:43056)
0812 02:48:15.716911 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 02:48:15.717024 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.193:39811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 02:48:15.717381 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 8619
2026-08-12T02:48:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3491 ms)
[----------] 6 tests from EncryptionPolicyTest (26014 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:15 dist-test-slave-ss4n krb5kdc[9263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:15 dist-test-slave-ss4n krb5kdc[9263](info): set up 2 sockets
Aug 12 02:48:15 dist-test-slave-ss4n krb5kdc[9263](info): commencing operation
krb5kdc: starting...
W20260812 02:48:17.125008 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.346s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:17 dist-test-slave-ss4n krb5kdc[9263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:17.520069 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44223
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:17.947382 9283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:17.947746 9283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:17.947846 9283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:17.957012 9283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:17.957188 9283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:17.957243 9283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:17.957340 9283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:17.957392 9283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:17.970714 9283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44223
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:17.972223 9283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:17.973903 9283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:17.989059 9289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:17.989138 9290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:17.988973 9292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:17.990303 9283 server_base.cc:1034] running on GCE node
I20260812 02:48:17.991243 9283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:17.992967 9283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:17.994253 9283 hybrid_clock.cc:648] HybridClock initialized: now 1786502897994186 us; error 54 us; skew 500 ppm
I20260812 02:48:17.994767 9283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:17 dist-test-slave-ss4n krb5kdc[9263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502897, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:17.998899 9283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:18.001132 9283 webserver.cc:460] Webserver started at http://127.0.0.1:38851/ using document root <none> and password file <none>
I20260812 02:48:18.002069 9283 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:18.002285 9283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:18.002673 9283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:18.006318 9283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "afdb002dbfc540bdbae2d26d33ac46a5"
format_stamp: "Formatted at 2026-08-12 02:48:18 on dist-test-slave-ss4n"
server_key: "57fc7839bfe1d4649384ed1f2eeabeda"
server_key_iv: "4b830c0a7bca46a68d0f56201d483129"
server_key_version: "encryptionkey@0"
I20260812 02:48:18.007437 9283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "afdb002dbfc540bdbae2d26d33ac46a5"
format_stamp: "Formatted at 2026-08-12 02:48:18 on dist-test-slave-ss4n"
server_key: "57fc7839bfe1d4649384ed1f2eeabeda"
server_key_iv: "4b830c0a7bca46a68d0f56201d483129"
server_key_version: "encryptionkey@0"
I20260812 02:48:18.014540 9283 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260812 02:48:18.019379 9301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:18.021418 9283 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:48:18.021799 9283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "afdb002dbfc540bdbae2d26d33ac46a5"
format_stamp: "Formatted at 2026-08-12 02:48:18 on dist-test-slave-ss4n"
server_key: "57fc7839bfe1d4649384ed1f2eeabeda"
server_key_iv: "4b830c0a7bca46a68d0f56201d483129"
server_key_version: "encryptionkey@0"
I20260812 02:48:18.022073 9283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:18.045358 9283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:18.051908 9283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:18.052333 9283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:18.077644 9283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44223
I20260812 02:48:18.077756 9357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44223 every 8 connection(s)
I20260812 02:48:18.079403 9283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:18.081717 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 9283
I20260812 02:48:18.081979 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:18.082993 18195 external_mini_cluster.cc:1428] Setting key 7dd6521395cbfe4eb9aec73504c094f0
I20260812 02:48:18.085348 9359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:48:18 dist-test-slave-ss4n krb5kdc[9263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 02:48:18.102689 9359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5: Bootstrap starting.
I20260812 02:48:18.106999 9359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:18.108662 9359 log.cc:824] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:18.112972 9359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5: No bootstrap required, opened a new log
I20260812 02:48:18.113699 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:18.085643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53971 (local address 127.0.0.1:44223)
0812 02:48:18.086630 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:18.086643 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:18.086691 (+ 48us) server_negotiation.cc:407] Connection header received
0812 02:48:18.088607 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:18.088646 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:18.089160 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:18.089655 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:18.091805 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:18.093599 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:18.095679 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:18.096675 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:18.096835 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:18.103098 (+ 6263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:18.103111 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:18.103131 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:18.103181 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:18.107637 (+ 4456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:18.109159 (+ 1522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:18.109168 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:18.109175 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:18.109477 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:18.110165 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:18.110173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:18.110179 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:18.110900 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:18.111159 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:18.111932 (+ 773us) server_negotiation.cc:299] Negotiation successful
0812 02:48:18.112418 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":460,"threads_started":1}
I20260812 02:48:18.118686 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:18.126055 9359 raft_consensus.cc:348] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdb002dbfc540bdbae2d26d33ac46a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44223 } }
I20260812 02:48:18.126668 9359 raft_consensus.cc:374] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:18.126884 9359 raft_consensus.cc:729] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afdb002dbfc540bdbae2d26d33ac46a5, State: Initialized, Role: FOLLOWER
I20260812 02:48:18.127764 9359 consensus_queue.cc:260] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdb002dbfc540bdbae2d26d33ac46a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44223 } }
I20260812 02:48:18.128161 9359 raft_consensus.cc:388] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:18.128329 9359 raft_consensus.cc:482] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:18.128561 9359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:18.132133 9359 raft_consensus.cc:504] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdb002dbfc540bdbae2d26d33ac46a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44223 } }
I20260812 02:48:18.132840 9359 leader_election.cc:302] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afdb002dbfc540bdbae2d26d33ac46a5; no voters:
I20260812 02:48:18.133839 9359 leader_election.cc:288] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:18.133972 9372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:18.136070 9372 raft_consensus.cc:686] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [term 1 LEADER]: Becoming Leader. State: Replica: afdb002dbfc540bdbae2d26d33ac46a5, State: Running, Role: LEADER
I20260812 02:48:18.136899 9359 sys_catalog.cc:564] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:18.136976 9372 consensus_queue.cc:237] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdb002dbfc540bdbae2d26d33ac46a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44223 } }
I20260812 02:48:18.141881 9373 sys_catalog.cc:455] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afdb002dbfc540bdbae2d26d33ac46a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdb002dbfc540bdbae2d26d33ac46a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44223 } } }
I20260812 02:48:18.142838 9373 sys_catalog.cc:458] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:18.145954 9374 sys_catalog.cc:455] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afdb002dbfc540bdbae2d26d33ac46a5. Latest consensus state: current_term: 1 leader_uuid: "afdb002dbfc540bdbae2d26d33ac46a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdb002dbfc540bdbae2d26d33ac46a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44223 } } }
I20260812 02:48:18.146435 9374 sys_catalog.cc:458] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:18.152184 9384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:18.152901 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:18.126344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48303 (local address 127.0.0.1:44223)
0812 02:48:18.126588 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:18.126595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:18.127383 (+ 788us) server_negotiation.cc:407] Connection header received
0812 02:48:18.127638 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:18.127643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:18.127740 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:18.127885 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:18.130008 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:18.131358 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:18.133182 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:18.134151 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:18.135279 (+ 1128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:18.138511 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:18.138519 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:18.138524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:18.138585 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:18.147188 (+ 8603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:18.148655 (+ 1467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:18.148659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:18.148661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:18.149030 (+ 369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:18.149787 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:18.149801 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:18.149806 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:18.150329 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:18.151590 (+ 1261us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:18.151899 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 02:48:18.152195 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 02:48:18.159109 9384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:18.171501 9384 catalog_manager.cc:1347] Generated new cluster ID: 10f88c2eea234eb78ee02bd2de4e3181
I20260812 02:48:18.171654 9384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:18.203398 9384 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:18.205188 9384 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:18.217031 9384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afdb002dbfc540bdbae2d26d33ac46a5: Generated new TSK 0
I20260812 02:48:18.218204 9384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:18.225800 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9283
I20260812 02:48:18.227360 9353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:18.338851 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 9283
W20260812 02:48:18.367771 9260 connection.cc:535] client connection to 127.0.0.1:44223 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44223): Connection reset by peer (error 104)
2026-08-12T02:48:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2611 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:18 dist-test-slave-ss4n krb5kdc[9397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:18 dist-test-slave-ss4n krb5kdc[9397](info): set up 2 sockets
Aug 12 02:48:18 dist-test-slave-ss4n krb5kdc[9397](info): commencing operation
krb5kdc: starting...
W20260812 02:48:19.719910 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:19 dist-test-slave-ss4n krb5kdc[9397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:20.122443 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43035
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:20.516984 9415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:20.517448 9415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:20.517714 9415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:20.526388 9415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:20.526597 9415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:20.526723 9415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:20.526851 9415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:20.526958 9415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:20.538962 9415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43035
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:20.540939 9415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:20.542620 9415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:20.556867 9425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:20.556905 9422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:20.556965 9423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:20.557858 9415 server_base.cc:1034] running on GCE node
I20260812 02:48:20.558907 9415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:20.560676 9415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:20.561949 9415 hybrid_clock.cc:648] HybridClock initialized: now 1786502900561888 us; error 39 us; skew 500 ppm
I20260812 02:48:20.562379 9415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:20 dist-test-slave-ss4n krb5kdc[9397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502900, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:20.570922 9415 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:20.572932 9415 webserver.cc:460] Webserver started at http://127.0.0.1:39295/ using document root <none> and password file <none>
I20260812 02:48:20.573844 9415 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:20.573980 9415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:20.574312 9415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:20.577939 9415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2"
format_stamp: "Formatted at 2026-08-12 02:48:20 on dist-test-slave-ss4n"
server_key: "9bf8a04dd1f86e571bdc38d1ce6b99c2"
server_key_iv: "7a8b3a5aa75214250bfd8bf1e0f4e8f5"
server_key_version: "encryptionkey@0"
I20260812 02:48:20.578833 9415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2"
format_stamp: "Formatted at 2026-08-12 02:48:20 on dist-test-slave-ss4n"
server_key: "9bf8a04dd1f86e571bdc38d1ce6b99c2"
server_key_iv: "7a8b3a5aa75214250bfd8bf1e0f4e8f5"
server_key_version: "encryptionkey@0"
I20260812 02:48:20.585373 9415 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260812 02:48:20.590065 9435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:20.591790 9415 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:48:20.591980 9415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2"
format_stamp: "Formatted at 2026-08-12 02:48:20 on dist-test-slave-ss4n"
server_key: "9bf8a04dd1f86e571bdc38d1ce6b99c2"
server_key_iv: "7a8b3a5aa75214250bfd8bf1e0f4e8f5"
server_key_version: "encryptionkey@0"
I20260812 02:48:20.592272 9415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:20.621943 9415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:20.626845 9415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:20.627190 9415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:20.649410 9415 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43035
I20260812 02:48:20.649470 9494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43035 every 8 connection(s)
I20260812 02:48:20.651103 9415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:20.652809 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 9415
I20260812 02:48:20.653070 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:20.653692 18195 external_mini_cluster.cc:1428] Setting key b1d28a67fbd2447d31f612fbe441b3e8
I20260812 02:48:20.656651 9495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:20.669390 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:20.655837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39029 (local address 127.0.0.1:43035)
0812 02:48:20.656788 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:20.656806 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:20.656863 (+ 57us) server_negotiation.cc:407] Connection header received
0812 02:48:20.658693 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:20.658723 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:20.659159 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:20.659586 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:20.660318 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:20.661854 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:20.663950 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:20.664735 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:20.664887 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:20.666417 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:20.666425 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:20.666441 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:20.667064 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:20.667156 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:20.667189 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:20.667332 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:20.667810 (+ 478us) server_negotiation.cc:299] Negotiation successful
0812 02:48:20.668064 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":437,"threads_started":1}
I20260812 02:48:20.671115 9495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2: Bootstrap starting.
I20260812 02:48:20.674536 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:20.674887 9495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:20.675994 9495 log.cc:824] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:20.679774 9495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2: No bootstrap required, opened a new log
I20260812 02:48:20.688673 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:20.680367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47563 (local address 127.0.0.1:43035)
0812 02:48:20.680610 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:20.680617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:20.681257 (+ 640us) server_negotiation.cc:407] Connection header received
0812 02:48:20.681440 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:20.681445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:20.681552 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:20.681673 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:20.682308 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:20.683582 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:20.685181 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:20.686001 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:20.686157 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:20.687648 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:20.687655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:20.687660 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:20.687691 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:20.687776 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:20.687807 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:20.687924 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:20.688206 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 02:48:20.688272 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 02:48:20.690236 9495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43035 } }
I20260812 02:48:20.690855 9495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:20.690995 9495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d09b963bb9544b4a3f0d6a0f31b52e2, State: Initialized, Role: FOLLOWER
I20260812 02:48:20.691721 9495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43035 } }
I20260812 02:48:20.691954 9495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:20.692094 9495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:20.692299 9495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:20.695274 9495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43035 } }
I20260812 02:48:20.695935 9495 leader_election.cc:302] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d09b963bb9544b4a3f0d6a0f31b52e2; no voters:
I20260812 02:48:20.696668 9495 leader_election.cc:288] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:20.696892 9506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:20.697796 9506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [term 1 LEADER]: Becoming Leader. State: Replica: 9d09b963bb9544b4a3f0d6a0f31b52e2, State: Running, Role: LEADER
I20260812 02:48:20.698485 9506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43035 } }
I20260812 02:48:20.699208 9495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:20.702262 9508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d09b963bb9544b4a3f0d6a0f31b52e2. Latest consensus state: current_term: 1 leader_uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43035 } } }
I20260812 02:48:20.702250 9507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d09b963bb9544b4a3f0d6a0f31b52e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43035 } } }
I20260812 02:48:20.702872 9507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:20.702847 9508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:20.704911 9515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:20.712580 9515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:20.725077 9515 catalog_manager.cc:1347] Generated new cluster ID: 9f99a1bd9b984f48a667179f9d445fef
I20260812 02:48:20.725310 9515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:20.744133 9515 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:20.745792 9515 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:20.756299 9515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d09b963bb9544b4a3f0d6a0f31b52e2: Generated new TSK 0
I20260812 02:48:20.757694 9515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:20.766708 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9415
W20260812 02:48:20.768311 9487 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39029
W20260812 02:48:20.769568 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9415: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:20.769807 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 9415
W20260812 02:48:20.795367 9395 connection.cc:535] client connection to 127.0.0.1:43035 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43035): Connection reset by peer (error 104)
2026-08-12T02:48:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:20 dist-test-slave-ss4n krb5kdc[9532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:20 dist-test-slave-ss4n krb5kdc[9532](info): set up 2 sockets
Aug 12 02:48:20 dist-test-slave-ss4n krb5kdc[9532](info): commencing operation
krb5kdc: starting...
W20260812 02:48:22.109376 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:22 dist-test-slave-ss4n krb5kdc[9532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:22.486466 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46455
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:22.881362 9549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:22.881794 9549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:22.881896 9549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:22.889880 9549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:22.890038 9549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:22.890093 9549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:22.890167 9549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:22.890223 9549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:22.902267 9549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46455
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:22.904155 9549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:22.905761 9549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:22.924296 9555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:22.924448 9557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:22.924466 9554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:22.926205 9549 server_base.cc:1034] running on GCE node
I20260812 02:48:22.927443 9549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:22.929382 9549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:22.930771 9549 hybrid_clock.cc:648] HybridClock initialized: now 1786502902930712 us; error 51 us; skew 500 ppm
I20260812 02:48:22.931286 9549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:22 dist-test-slave-ss4n krb5kdc[9532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502902, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:22.935611 9549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:22.937927 9549 webserver.cc:460] Webserver started at http://127.0.0.1:33197/ using document root <none> and password file <none>
I20260812 02:48:22.938808 9549 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:22.939044 9549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:22.939438 9549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:22.943116 9549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "47677b6378d34bf19ebb4adedfdfdfe9"
format_stamp: "Formatted at 2026-08-12 02:48:22 on dist-test-slave-ss4n"
server_key: "e90dcc0b2bc0b230507cc8e9c0fd0106"
server_key_iv: "00e9f8e12c93d8186d87477471baa728"
server_key_version: "encryptionkey@0"
I20260812 02:48:22.944124 9549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "47677b6378d34bf19ebb4adedfdfdfe9"
format_stamp: "Formatted at 2026-08-12 02:48:22 on dist-test-slave-ss4n"
server_key: "e90dcc0b2bc0b230507cc8e9c0fd0106"
server_key_iv: "00e9f8e12c93d8186d87477471baa728"
server_key_version: "encryptionkey@0"
I20260812 02:48:22.950665 9549 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260812 02:48:22.955226 9564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:22.956933 9549 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:48:22.957211 9549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "47677b6378d34bf19ebb4adedfdfdfe9"
format_stamp: "Formatted at 2026-08-12 02:48:22 on dist-test-slave-ss4n"
server_key: "e90dcc0b2bc0b230507cc8e9c0fd0106"
server_key_iv: "00e9f8e12c93d8186d87477471baa728"
server_key_version: "encryptionkey@0"
I20260812 02:48:22.957458 9549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:22.964911 9549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:22.970938 9549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:22.971335 9549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:22.995651 9549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46455
I20260812 02:48:22.995679 9630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46455 every 8 connection(s)
I20260812 02:48:22.997390 9549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:23.002010 9633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:23.006058 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 9549
I20260812 02:48:23.006274 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:23.006809 18195 external_mini_cluster.cc:1428] Setting key c327e62101ea981a7a56e2c3ead72b2c
I20260812 02:48:23.018743 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:23.009372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41587 (local address 127.0.0.1:46455)
0812 02:48:23.010575 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:23.010589 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:23.010646 (+ 57us) server_negotiation.cc:407] Connection header received
0812 02:48:23.013710 (+ 3064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:23.013745 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:23.014252 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:23.014754 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:23.016175 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:23.016184 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:23.016201 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:23.016222 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:23.016308 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:23.016339 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:23.016596 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:23.016909 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 02:48:23.017216 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":916,"thread_start_us":478,"threads_started":1}
I20260812 02:48:23.018904 9633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9: Bootstrap starting.
I20260812 02:48:23.023219 9633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:23.023859 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:23.024652 9633 log.cc:824] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:23.028431 9633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9: No bootstrap required, opened a new log
I20260812 02:48:23.032917 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:23.029649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58751 (local address 127.0.0.1:46455)
0812 02:48:23.029905 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:23.029917 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:23.030479 (+ 562us) server_negotiation.cc:407] Connection header received
0812 02:48:23.030697 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:23.030702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:23.030766 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:23.030871 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:23.032048 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:23.032052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:23.032056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:23.032080 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:23.032149 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:23.032167 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:23.032279 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:23.032420 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 02:48:23.032473 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:48:23.038021 9633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47677b6378d34bf19ebb4adedfdfdfe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46455 } }
I20260812 02:48:23.038509 9633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:23.038626 9633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47677b6378d34bf19ebb4adedfdfdfe9, State: Initialized, Role: FOLLOWER
I20260812 02:48:23.039297 9633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47677b6378d34bf19ebb4adedfdfdfe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46455 } }
I20260812 02:48:23.039625 9633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:23.039971 9633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:23.040217 9633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:23.044158 9633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47677b6378d34bf19ebb4adedfdfdfe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46455 } }
I20260812 02:48:23.044885 9633 leader_election.cc:302] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47677b6378d34bf19ebb4adedfdfdfe9; no voters:
I20260812 02:48:23.045769 9633 leader_election.cc:288] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:23.045948 9644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:23.047107 9644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [term 1 LEADER]: Becoming Leader. State: Replica: 47677b6378d34bf19ebb4adedfdfdfe9, State: Running, Role: LEADER
I20260812 02:48:23.047838 9644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47677b6378d34bf19ebb4adedfdfdfe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46455 } }
I20260812 02:48:23.050395 9644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47677b6378d34bf19ebb4adedfdfdfe9. Latest consensus state: current_term: 1 leader_uuid: "47677b6378d34bf19ebb4adedfdfdfe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47677b6378d34bf19ebb4adedfdfdfe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46455 } } }
I20260812 02:48:23.050843 9644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:23.051496 9646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47677b6378d34bf19ebb4adedfdfdfe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47677b6378d34bf19ebb4adedfdfdfe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46455 } } }
I20260812 02:48:23.052039 9646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:23.055068 9633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:23.055347 9649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:23.063350 9649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:23.075379 9649 catalog_manager.cc:1347] Generated new cluster ID: e661aea5939c429fabf931f698891750
I20260812 02:48:23.075534 9649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:23.089694 9649 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:23.090266 9649 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:23.106056 9649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47677b6378d34bf19ebb4adedfdfdfe9: Generated new TSK 0
I20260812 02:48:23.107057 9649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:23.125424 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9549
W20260812 02:48:23.126960 9622 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41587
W20260812 02:48:23.127823 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9549: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:23.127998 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 9549
2026-08-12T02:48:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:23 dist-test-slave-ss4n krb5kdc[9667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:23 dist-test-slave-ss4n krb5kdc[9667](info): set up 2 sockets
Aug 12 02:48:23 dist-test-slave-ss4n krb5kdc[9667](info): commencing operation
krb5kdc: starting...
W20260812 02:48:24.506757 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:24 dist-test-slave-ss4n krb5kdc[9667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:24.908484 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:25.336301 9681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:25.336660 9681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:25.336825 9681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:25.345394 9681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:25.345654 9681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:25.345753 9681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:25.345851 9681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:25.345912 9681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:25.360569 9681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:25.362772 9681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:25.364698 9681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:25.381322 9688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:25.381385 9687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:25.381326 9690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:25.382995 9681 server_base.cc:1034] running on GCE node
I20260812 02:48:25.384698 9681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:25.387466 9681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:25.391078 9681 hybrid_clock.cc:648] HybridClock initialized: now 1786502905390784 us; error 271 us; skew 500 ppm
I20260812 02:48:25.391953 9681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:25 dist-test-slave-ss4n krb5kdc[9667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502905, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:25.396061 9681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:25.398448 9681 webserver.cc:460] Webserver started at http://127.0.0.1:44823/ using document root <none> and password file <none>
I20260812 02:48:25.399719 9681 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:25.400086 9681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:25.400794 9681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:25.406152 9681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa"
format_stamp: "Formatted at 2026-08-12 02:48:25 on dist-test-slave-ss4n"
server_key: "b73e79757be5a69c544a7fd81bbb1383"
server_key_iv: "df8d5edd14ab0e59e04ac7aafce040d0"
server_key_version: "encryptionkey@0"
I20260812 02:48:25.407232 9681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa"
format_stamp: "Formatted at 2026-08-12 02:48:25 on dist-test-slave-ss4n"
server_key: "b73e79757be5a69c544a7fd81bbb1383"
server_key_iv: "df8d5edd14ab0e59e04ac7aafce040d0"
server_key_version: "encryptionkey@0"
I20260812 02:48:25.414238 9681 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260812 02:48:25.419050 9697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:25.421191 9681 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:48:25.421404 9681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa"
format_stamp: "Formatted at 2026-08-12 02:48:25 on dist-test-slave-ss4n"
server_key: "b73e79757be5a69c544a7fd81bbb1383"
server_key_iv: "df8d5edd14ab0e59e04ac7aafce040d0"
server_key_version: "encryptionkey@0"
I20260812 02:48:25.421756 9681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:25.455751 9681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:25.462654 9681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:25.463034 9681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:25.487120 9681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44245
I20260812 02:48:25.487210 9756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44245 every 8 connection(s)
I20260812 02:48:25.488845 9681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:25.491739 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 9681
I20260812 02:48:25.491966 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:25.492520 18195 external_mini_cluster.cc:1428] Setting key 9d14535f51cf8cb67e6055f2319139a9
I20260812 02:48:25.493304 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:48:25 dist-test-slave-ss4n krb5kdc[9667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 02:48:25.510222 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa: Bootstrap starting.
I20260812 02:48:25.513967 9757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:25.515233 9757 log.cc:824] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:25.519057 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa: No bootstrap required, opened a new log
I20260812 02:48:25.519951 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:25.494502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60545 (local address 127.0.0.1:44245)
0812 02:48:25.495356 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:25.495369 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:25.495418 (+ 49us) server_negotiation.cc:407] Connection header received
0812 02:48:25.497121 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:25.497148 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:25.497701 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:25.498116 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:25.500268 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:25.502178 (+ 1910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:25.504152 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:25.505029 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:25.505201 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:25.511383 (+ 6182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:25.511392 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:25.511410 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:25.511463 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:25.515051 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:25.516012 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:25.516020 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:25.516028 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:25.516304 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:25.516774 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:25.516781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:25.516790 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:25.517371 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:25.517754 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:25.518320 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 02:48:25.518805 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":395,"threads_started":1}
I20260812 02:48:25.524603 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:25.528308 9757 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44245 } }
I20260812 02:48:25.528869 9757 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:25.528998 9757 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac33ccc09a7b456a96e3ea05ec7c6dfa, State: Initialized, Role: FOLLOWER
I20260812 02:48:25.529809 9757 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44245 } }
I20260812 02:48:25.530104 9757 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:25.530330 9757 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:25.530544 9757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:25.534299 9757 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44245 } }
I20260812 02:48:25.534931 9757 leader_election.cc:302] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac33ccc09a7b456a96e3ea05ec7c6dfa; no voters:
I20260812 02:48:25.535723 9757 leader_election.cc:288] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:25.535854 9770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:25.537230 9770 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [term 1 LEADER]: Becoming Leader. State: Replica: ac33ccc09a7b456a96e3ea05ec7c6dfa, State: Running, Role: LEADER
I20260812 02:48:25.537963 9770 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44245 } }
I20260812 02:48:25.538443 9757 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:25.541309 9771 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44245 } } }
I20260812 02:48:25.541893 9771 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:25.546098 9772 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac33ccc09a7b456a96e3ea05ec7c6dfa. Latest consensus state: current_term: 1 leader_uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac33ccc09a7b456a96e3ea05ec7c6dfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44245 } } }
I20260812 02:48:25.547267 9772 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:25.550423 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:25.529992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49775 (local address 127.0.0.1:44245)
0812 02:48:25.530250 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:25.530258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:25.530797 (+ 539us) server_negotiation.cc:407] Connection header received
0812 02:48:25.531011 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:25.531015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:25.531098 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:25.531189 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:25.533626 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:25.534436 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:25.536023 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:25.536826 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:25.536958 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:25.540134 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:25.540141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:25.540145 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:25.540197 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:25.544614 (+ 4417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:25.546003 (+ 1389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:25.546012 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:25.546018 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:25.546337 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:25.547039 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:25.547046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:25.547053 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:25.547582 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:25.549392 (+ 1810us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:25.549630 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 02:48:25.549972 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:48:25.551301 9775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:25.556901 9775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:25.569758 9775 catalog_manager.cc:1347] Generated new cluster ID: 34821d495cfa4426a022fbf65376a25c
I20260812 02:48:25.569904 9775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:25.583735 9775 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:25.585464 9775 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:25.598264 9775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac33ccc09a7b456a96e3ea05ec7c6dfa: Generated new TSK 0
I20260812 02:48:25.599586 9775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:25.627105 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9681
I20260812 02:48:25.628729 9751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:25.738265 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 9681
2026-08-12T02:48:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:25 dist-test-slave-ss4n krb5kdc[9795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:25 dist-test-slave-ss4n krb5kdc[9795](info): set up 2 sockets
Aug 12 02:48:25 dist-test-slave-ss4n krb5kdc[9795](info): commencing operation
krb5kdc: starting...
W20260812 02:48:27.119107 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:27 dist-test-slave-ss4n krb5kdc[9795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:27.531327 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:27.945212 9809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:27.945736 9809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:27.945919 9809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:27.955226 9809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:27.955440 9809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:27.955564 9809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:27.955650 9809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:27.955739 9809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:27.967828 9809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:27.969425 9809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:27.971060 9809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:27.986929 9817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:27.986958 9820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:27.987042 9815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:27.987697 9809 server_base.cc:1034] running on GCE node
I20260812 02:48:27.988765 9809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:27.990705 9809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:27.992057 9809 hybrid_clock.cc:648] HybridClock initialized: now 1786502907991957 us; error 47 us; skew 500 ppm
I20260812 02:48:27.992590 9809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:27 dist-test-slave-ss4n krb5kdc[9795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:27.996780 9809 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:27.998703 9809 webserver.cc:460] Webserver started at http://127.0.0.1:37321/ using document root <none> and password file <none>
I20260812 02:48:27.999516 9809 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:27.999691 9809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:28.000046 9809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:28.003583 9809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "6527bf00d46940c2a3047b4906473784"
format_stamp: "Formatted at 2026-08-12 02:48:27 on dist-test-slave-ss4n"
server_key: "be64a29cbc5cae78cc6d8c849b6311f8"
server_key_iv: "e9b27eb479dfde23fec3c8658a4a0e64"
server_key_version: "encryptionkey@0"
I20260812 02:48:28.004524 9809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "6527bf00d46940c2a3047b4906473784"
format_stamp: "Formatted at 2026-08-12 02:48:27 on dist-test-slave-ss4n"
server_key: "be64a29cbc5cae78cc6d8c849b6311f8"
server_key_iv: "e9b27eb479dfde23fec3c8658a4a0e64"
server_key_version: "encryptionkey@0"
I20260812 02:48:28.010994 9809 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 02:48:28.015413 9828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:28.017100 9809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:48:28.017379 9809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "6527bf00d46940c2a3047b4906473784"
format_stamp: "Formatted at 2026-08-12 02:48:27 on dist-test-slave-ss4n"
server_key: "be64a29cbc5cae78cc6d8c849b6311f8"
server_key_iv: "e9b27eb479dfde23fec3c8658a4a0e64"
server_key_version: "encryptionkey@0"
I20260812 02:48:28.017704 9809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:28.032897 9809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:28.040566 9809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:28.041056 9809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:28.064952 9809 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46585
I20260812 02:48:28.064989 9894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46585 every 8 connection(s)
I20260812 02:48:28.067196 9809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:28.072012 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 9809
I20260812 02:48:28.072199 9895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:28.072202 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:28.072767 18195 external_mini_cluster.cc:1428] Setting key 944e88b696768452e647a6aeb1493bd2
I20260812 02:48:28.087841 9895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784: Bootstrap starting.
I20260812 02:48:28.089718 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:28.075288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48627 (local address 127.0.0.1:46585)
0812 02:48:28.076226 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:28.076240 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:28.076295 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:48:28.078871 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:28.078903 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:28.079435 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:28.079944 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:28.080619 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:28.082167 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:28.083752 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:28.084623 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:28.084902 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:28.086655 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:28.086665 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:28.086685 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:28.087419 (+ 734us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:28.087531 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:28.087565 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:28.087737 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:28.088048 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 02:48:28.088442 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":411,"threads_started":1}
I20260812 02:48:28.092047 9895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:28.093791 9895 log.cc:824] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:28.095593 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:28.098320 9895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784: No bootstrap required, opened a new log
I20260812 02:48:28.110865 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:28.102137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35297 (local address 127.0.0.1:46585)
0812 02:48:28.102478 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:28.102489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:28.102943 (+ 454us) server_negotiation.cc:407] Connection header received
0812 02:48:28.103204 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:28.103209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:28.103287 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:28.103398 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:28.104143 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:28.105022 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:28.107438 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:28.108037 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:28.108163 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:28.109763 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:28.109770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:28.109774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:28.109800 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:28.109869 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:28.109893 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:28.110021 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:28.110215 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 02:48:28.110316 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 02:48:28.110812 9895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6527bf00d46940c2a3047b4906473784" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } }
I20260812 02:48:28.111431 9895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:28.111589 9895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6527bf00d46940c2a3047b4906473784, State: Initialized, Role: FOLLOWER
I20260812 02:48:28.112304 9895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6527bf00d46940c2a3047b4906473784" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } }
I20260812 02:48:28.112634 9895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:28.112824 9895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:28.113031 9895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:28.115645 9895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6527bf00d46940c2a3047b4906473784" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } }
I20260812 02:48:28.116341 9895 leader_election.cc:302] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6527bf00d46940c2a3047b4906473784; no voters:
I20260812 02:48:28.117122 9895 leader_election.cc:288] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:28.117298 9906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:28.118258 9906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [term 1 LEADER]: Becoming Leader. State: Replica: 6527bf00d46940c2a3047b4906473784, State: Running, Role: LEADER
I20260812 02:48:28.118944 9906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6527bf00d46940c2a3047b4906473784" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } }
I20260812 02:48:28.119860 9895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:28.122380 9907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6527bf00d46940c2a3047b4906473784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6527bf00d46940c2a3047b4906473784" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } } }
I20260812 02:48:28.122471 9908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6527bf00d46940c2a3047b4906473784. Latest consensus state: current_term: 1 leader_uuid: "6527bf00d46940c2a3047b4906473784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6527bf00d46940c2a3047b4906473784" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46585 } } }
I20260812 02:48:28.123076 9907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:28.123131 9908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:28.127586 9914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:28.133358 9914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:28.145720 9914 catalog_manager.cc:1347] Generated new cluster ID: a59f7f9fcf4f41bebdb92df9982ffa6f
I20260812 02:48:28.145888 9914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:28.189054 9914 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:28.191516 9914 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:28.207494 9914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6527bf00d46940c2a3047b4906473784: Generated new TSK 0
I20260812 02:48:28.208845 9914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:28.231737 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9809
W20260812 02:48:28.233815 9890 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48627
W20260812 02:48:28.234836 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9809: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:28.235039 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 9809
2026-08-12T02:48:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:28 dist-test-slave-ss4n krb5kdc[9933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:28 dist-test-slave-ss4n krb5kdc[9933](info): set up 2 sockets
Aug 12 02:48:28 dist-test-slave-ss4n krb5kdc[9933](info): commencing operation
krb5kdc: starting...
W20260812 02:48:29.543830 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:29 dist-test-slave-ss4n krb5kdc[9933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:29.938836 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:30.320246 9950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:30.320611 9950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:30.320717 9950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:30.328471 9950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:30.328639 9950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:30.328689 9950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:30.328747 9950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:30.328786 9950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:30.339344 9950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:30.340752 9950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:30.342199 9950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:30.356124 9955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:30.356119 9956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:30.356163 9958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:30.357721 9950 server_base.cc:1034] running on GCE node
I20260812 02:48:30.358721 9950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:30.360548 9950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:30.361821 9950 hybrid_clock.cc:648] HybridClock initialized: now 1786502910361769 us; error 44 us; skew 500 ppm
I20260812 02:48:30.362279 9950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:30 dist-test-slave-ss4n krb5kdc[9933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:30.366112 9950 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:30.367981 9950 webserver.cc:460] Webserver started at http://127.0.0.1:44659/ using document root <none> and password file <none>
I20260812 02:48:30.368752 9950 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:30.368912 9950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:30.369248 9950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:30.372771 9950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "e27c6060d1744a77bec54ca9b9299361"
format_stamp: "Formatted at 2026-08-12 02:48:30 on dist-test-slave-ss4n"
server_key: "7a399e690ba9fcb5284d8726aba001fd"
server_key_iv: "a802ef1c5ad9177b9067ae7bf6c27893"
server_key_version: "encryptionkey@0"
I20260812 02:48:30.373766 9950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "e27c6060d1744a77bec54ca9b9299361"
format_stamp: "Formatted at 2026-08-12 02:48:30 on dist-test-slave-ss4n"
server_key: "7a399e690ba9fcb5284d8726aba001fd"
server_key_iv: "a802ef1c5ad9177b9067ae7bf6c27893"
server_key_version: "encryptionkey@0"
I20260812 02:48:30.379971 9950 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:48:30.384320 9969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:30.386463 9950 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 02:48:30.386723 9950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "e27c6060d1744a77bec54ca9b9299361"
format_stamp: "Formatted at 2026-08-12 02:48:30 on dist-test-slave-ss4n"
server_key: "7a399e690ba9fcb5284d8726aba001fd"
server_key_iv: "a802ef1c5ad9177b9067ae7bf6c27893"
server_key_version: "encryptionkey@0"
I20260812 02:48:30.386982 9950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:30.394630 9950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:30.400440 9950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:30.400813 9950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:30.422843 9950 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34255
I20260812 02:48:30.422914 10032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34255 every 8 connection(s)
I20260812 02:48:30.424480 9950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:30.427629 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 9950
I20260812 02:48:30.427913 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:30.428609 18195 external_mini_cluster.cc:1428] Setting key 5013b4432183d69f0267ad0c818a2bd7
I20260812 02:48:30.430311 10033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:30.438290 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:30.430946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51517 (local address 127.0.0.1:34255)
0812 02:48:30.431886 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:30.431900 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:30.431945 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:48:30.433741 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:30.433769 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:30.434182 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:30.434579 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:30.435999 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:30.436010 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:30.436032 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:30.436061 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:30.436170 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:30.436212 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:30.436433 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:30.436642 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 02:48:30.436987 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":448,"threads_started":1}
I20260812 02:48:30.443466 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:30.445282 10033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361: Bootstrap starting.
I20260812 02:48:30.449087 10033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:30.450685 10033 log.cc:824] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:30.452389 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:30.449099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34429 (local address 127.0.0.1:34255)
0812 02:48:30.449326 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:30.449334 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:30.449861 (+ 527us) server_negotiation.cc:407] Connection header received
0812 02:48:30.450062 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:30.450068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:30.450141 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:30.450263 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:30.451503 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:30.451513 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:30.451521 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:30.451554 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:30.451603 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:30.451622 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:30.451801 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:30.451955 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 02:48:30.452051 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:48:30.454311 10033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361: No bootstrap required, opened a new log
I20260812 02:48:30.463220 10033 raft_consensus.cc:348] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27c6060d1744a77bec54ca9b9299361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } }
I20260812 02:48:30.463670 10033 raft_consensus.cc:374] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:30.463835 10033 raft_consensus.cc:729] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e27c6060d1744a77bec54ca9b9299361, State: Initialized, Role: FOLLOWER
I20260812 02:48:30.464699 10033 consensus_queue.cc:260] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27c6060d1744a77bec54ca9b9299361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } }
I20260812 02:48:30.465018 10033 raft_consensus.cc:388] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:30.465207 10033 raft_consensus.cc:482] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:30.465432 10033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:30.468537 10033 raft_consensus.cc:504] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27c6060d1744a77bec54ca9b9299361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } }
I20260812 02:48:30.469228 10033 leader_election.cc:302] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e27c6060d1744a77bec54ca9b9299361; no voters:
I20260812 02:48:30.470011 10033 leader_election.cc:288] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:30.470263 10045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:30.471139 10045 raft_consensus.cc:686] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [term 1 LEADER]: Becoming Leader. State: Replica: e27c6060d1744a77bec54ca9b9299361, State: Running, Role: LEADER
I20260812 02:48:30.471875 10045 consensus_queue.cc:237] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27c6060d1744a77bec54ca9b9299361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } }
I20260812 02:48:30.472756 10033 sys_catalog.cc:564] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:30.475277 10047 sys_catalog.cc:455] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e27c6060d1744a77bec54ca9b9299361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27c6060d1744a77bec54ca9b9299361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } } }
I20260812 02:48:30.475239 10048 sys_catalog.cc:455] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e27c6060d1744a77bec54ca9b9299361. Latest consensus state: current_term: 1 leader_uuid: "e27c6060d1744a77bec54ca9b9299361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27c6060d1744a77bec54ca9b9299361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } } }
I20260812 02:48:30.475970 10047 sys_catalog.cc:458] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:30.475970 10048 sys_catalog.cc:458] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:30.478611 10052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:30.484714 10052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:30.496183 10052 catalog_manager.cc:1347] Generated new cluster ID: b8872f7461c64ce08cb0047208a3e59d
I20260812 02:48:30.496322 10052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:30.514043 10052 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:30.514606 10052 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:30.523176 10052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e27c6060d1744a77bec54ca9b9299361: Generated new TSK 0
I20260812 02:48:30.524248 10052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:30.535852 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9950
W20260812 02:48:30.537506 10026 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51517
W20260812 02:48:30.538441 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 9950: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:30.538621 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 9950
2026-08-12T02:48:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:30 dist-test-slave-ss4n krb5kdc[10068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:30 dist-test-slave-ss4n krb5kdc[10068](info): set up 2 sockets
Aug 12 02:48:30 dist-test-slave-ss4n krb5kdc[10068](info): commencing operation
krb5kdc: starting...
W20260812 02:48:31.888619 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:32 dist-test-slave-ss4n krb5kdc[10068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:32.286509 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:32.663262 10081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:32.663626 10081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:32.663899 10081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:32.674465 10081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:32.674697 10081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:32.674813 10081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:32.674914 10081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:32.675009 10081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:32.687445 10081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:32.688938 10081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:32.690519 10081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:32.704823 10090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:32.704869 10086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:32.704835 10087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:32.706550 10081 server_base.cc:1034] running on GCE node
I20260812 02:48:32.707592 10081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:32.709410 10081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:32.710804 10081 hybrid_clock.cc:648] HybridClock initialized: now 1786502912710728 us; error 83 us; skew 500 ppm
I20260812 02:48:32.711344 10081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:32 dist-test-slave-ss4n krb5kdc[10068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502912, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:32.715615 10081 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:32.718061 10081 webserver.cc:460] Webserver started at http://127.0.0.1:41469/ using document root <none> and password file <none>
I20260812 02:48:32.718883 10081 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:32.719022 10081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:32.719566 10081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:32.723699 10081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "c4e31f5db08a448895581b32000a2621"
format_stamp: "Formatted at 2026-08-12 02:48:32 on dist-test-slave-ss4n"
server_key: "f69274669b39cee84ed6840b8d4a9fab"
server_key_iv: "d68272611565c35500ce7d636cd23b5e"
server_key_version: "encryptionkey@0"
I20260812 02:48:32.724727 10081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "c4e31f5db08a448895581b32000a2621"
format_stamp: "Formatted at 2026-08-12 02:48:32 on dist-test-slave-ss4n"
server_key: "f69274669b39cee84ed6840b8d4a9fab"
server_key_iv: "d68272611565c35500ce7d636cd23b5e"
server_key_version: "encryptionkey@0"
I20260812 02:48:32.731318 10081 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 02:48:32.735472 10100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:32.737295 10081 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 02:48:32.737564 10081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "c4e31f5db08a448895581b32000a2621"
format_stamp: "Formatted at 2026-08-12 02:48:32 on dist-test-slave-ss4n"
server_key: "f69274669b39cee84ed6840b8d4a9fab"
server_key_iv: "d68272611565c35500ce7d636cd23b5e"
server_key_version: "encryptionkey@0"
I20260812 02:48:32.737823 10081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:32.755035 10081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:32.760248 10081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:32.760540 10081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:32.785204 10081 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37067
I20260812 02:48:32.785254 10158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37067 every 8 connection(s)
I20260812 02:48:32.787045 10081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:32.792061 10159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:32.796525 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 10081
I20260812 02:48:32.796813 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:32.797293 18195 external_mini_cluster.cc:1428] Setting key dcb85e4cb113e4c264fcae21a760b581
I20260812 02:48:32.807372 10159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621: Bootstrap starting.
I20260812 02:48:32.812033 10159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:32.813738 10159 log.cc:824] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:48:32 dist-test-slave-ss4n krb5kdc[10068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 02:48:32.817327 10159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621: No bootstrap required, opened a new log
I20260812 02:48:32.825309 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:32.799586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:54345 (local address 127.0.0.1:37067)
0812 02:48:32.800459 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:32.800472 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:32.800521 (+ 49us) server_negotiation.cc:407] Connection header received
0812 02:48:32.802390 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:32.802422 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:32.802930 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:32.803349 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:32.805472 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:32.807085 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:32.809066 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:32.810130 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:32.810298 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:32.816003 (+ 5705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:32.816013 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:32.816032 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:32.816077 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:32.819871 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:32.820996 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:32.821004 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:32.821013 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:32.821234 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:32.821908 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:32.821912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:32.821914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:32.822641 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:32.823050 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:32.823678 (+ 628us) server_negotiation.cc:299] Negotiation successful
0812 02:48:32.824057 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":404,"threads_started":1}
I20260812 02:48:32.826267 10159 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e31f5db08a448895581b32000a2621" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37067 } }
I20260812 02:48:32.826789 10159 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:32.826930 10159 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e31f5db08a448895581b32000a2621, State: Initialized, Role: FOLLOWER
I20260812 02:48:32.827601 10159 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e31f5db08a448895581b32000a2621" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37067 } }
I20260812 02:48:32.827917 10159 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:32.828088 10159 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:32.828286 10159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:32.831012 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:32.831688 10159 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e31f5db08a448895581b32000a2621" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37067 } }
I20260812 02:48:32.832304 10159 leader_election.cc:302] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e31f5db08a448895581b32000a2621; no voters:
I20260812 02:48:32.833132 10159 leader_election.cc:288] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:32.833258 10164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:32.836257 10159 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:32.838305 10164 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [term 1 LEADER]: Becoming Leader. State: Replica: c4e31f5db08a448895581b32000a2621, State: Running, Role: LEADER
I20260812 02:48:32.839389 10164 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e31f5db08a448895581b32000a2621" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37067 } }
I20260812 02:48:32.843567 10165 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e31f5db08a448895581b32000a2621" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e31f5db08a448895581b32000a2621" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37067 } } }
I20260812 02:48:32.844127 10165 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:32.844821 10166 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e31f5db08a448895581b32000a2621. Latest consensus state: current_term: 1 leader_uuid: "c4e31f5db08a448895581b32000a2621" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e31f5db08a448895581b32000a2621" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37067 } } }
I20260812 02:48:32.845433 10166 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:32.854712 10181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:32.861558 10181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:32.870049 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:32.843576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:47267 (local address 127.0.0.1:37067)
0812 02:48:32.844379 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:32.844386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:32.845892 (+ 1506us) server_negotiation.cc:407] Connection header received
0812 02:48:32.846204 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:32.846212 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:32.846322 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:32.846422 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:32.849521 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:32.851305 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:32.854112 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:32.854970 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:32.855118 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:32.858988 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:32.858995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:32.858998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:32.859038 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:32.863475 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:32.865191 (+ 1716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:32.865201 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:32.865208 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:32.865500 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:32.866380 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:32.866388 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:32.866394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:32.867396 (+ 1002us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:32.867810 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:32.869348 (+ 1538us) server_negotiation.cc:299] Negotiation successful
0812 02:48:32.869690 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643}
I20260812 02:48:32.873965 10181 catalog_manager.cc:1347] Generated new cluster ID: 9b1a25f00e5d414789a64e2ae143f2a9
I20260812 02:48:32.874120 10181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:32.888063 10181 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:32.890399 10181 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:32.908692 10181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e31f5db08a448895581b32000a2621: Generated new TSK 0
I20260812 02:48:32.910039 10181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:32.925590 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10081
I20260812 02:48:32.927017 10154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:33.025743 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 10081
W20260812 02:48:33.050995 10066 connection.cc:535] client connection to 127.0.0.1:37067 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37067): Connection reset by peer (error 104)
2026-08-12T02:48:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:33 dist-test-slave-ss4n krb5kdc[10198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:33 dist-test-slave-ss4n krb5kdc[10198](info): set up 2 sockets
Aug 12 02:48:33 dist-test-slave-ss4n krb5kdc[10198](info): commencing operation
krb5kdc: starting...
W20260812 02:48:34.336225 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:34 dist-test-slave-ss4n krb5kdc[10198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:34.719000 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:35.090888 10214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:35.091243 10214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:35.091347 10214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:35.099113 10214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:35.099272 10214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:35.099323 10214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:35.099380 10214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:35.099503 10214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:35.110321 10214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:35.111738 10214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:35.113081 10214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:35.127904 10220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:35.128010 10219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:35.128017 10222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:35.129060 10214 server_base.cc:1034] running on GCE node
I20260812 02:48:35.130113 10214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:35.131904 10214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:35.133172 10214 hybrid_clock.cc:648] HybridClock initialized: now 1786502915133121 us; error 32 us; skew 500 ppm
I20260812 02:48:35.133780 10214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:35 dist-test-slave-ss4n krb5kdc[10198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502915, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:35.137550 10214 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:35.139480 10214 webserver.cc:460] Webserver started at http://127.0.0.1:46301/ using document root <none> and password file <none>
I20260812 02:48:35.140271 10214 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:35.140422 10214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:35.140781 10214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:35.144227 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd"
format_stamp: "Formatted at 2026-08-12 02:48:35 on dist-test-slave-ss4n"
server_key: "24e476905c9d767fac60fa18b1899ee6"
server_key_iv: "8dd50508b9b6216466dd460bbe68c4d2"
server_key_version: "encryptionkey@0"
I20260812 02:48:35.145174 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd"
format_stamp: "Formatted at 2026-08-12 02:48:35 on dist-test-slave-ss4n"
server_key: "24e476905c9d767fac60fa18b1899ee6"
server_key_iv: "8dd50508b9b6216466dd460bbe68c4d2"
server_key_version: "encryptionkey@0"
I20260812 02:48:35.151542 10214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 02:48:35.156265 10229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:35.157946 10214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:48:35.158167 10214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd"
format_stamp: "Formatted at 2026-08-12 02:48:35 on dist-test-slave-ss4n"
server_key: "24e476905c9d767fac60fa18b1899ee6"
server_key_iv: "8dd50508b9b6216466dd460bbe68c4d2"
server_key_version: "encryptionkey@0"
I20260812 02:48:35.158401 10214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:35.172915 10214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:35.178018 10214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:35.178349 10214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:35.199841 10214 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46875
I20260812 02:48:35.199892 10291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46875 every 8 connection(s)
I20260812 02:48:35.201370 10214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:35.205824 10293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:35.206517 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 10214
I20260812 02:48:35.206732 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:35.207342 18195 external_mini_cluster.cc:1428] Setting key 0ece5cba76b75c55864ad0329ba3b4cc
I20260812 02:48:35.222496 10293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd: Bootstrap starting.
I20260812 02:48:35.224489 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:35.209682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:35219 (local address 127.0.0.1:46875)
0812 02:48:35.210662 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:35.210675 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:35.210725 (+ 50us) server_negotiation.cc:407] Connection header received
0812 02:48:35.213123 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:35.213155 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:35.213778 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:35.214195 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:35.215016 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:35.216559 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:35.218941 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:35.219782 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:35.219930 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:35.221755 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:35.221765 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:35.221783 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:35.222438 (+ 655us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:35.222547 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:35.222578 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:35.222824 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:35.223178 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 02:48:35.223391 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":472,"threads_started":1}
I20260812 02:48:35.226752 10293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:35.228350 10293 log.cc:824] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:35.229660 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:35.231927 10293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd: No bootstrap required, opened a new log
I20260812 02:48:35.241078 10293 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260812 02:48:35.241622 10293 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:35.241777 10293 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5c5b5beb9f44a4a790f287aacfe9fd, State: Initialized, Role: FOLLOWER
I20260812 02:48:35.242689 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:35.235035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:35607 (local address 127.0.0.1:46875)
0812 02:48:35.235284 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:35.235294 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:35.235760 (+ 466us) server_negotiation.cc:407] Connection header received
0812 02:48:35.236041 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:35.236046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:35.236129 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:35.236285 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:35.236862 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:35.237853 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:35.239467 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:35.240007 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:35.240171 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:35.241733 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:35.241765 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:35.241771 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:35.241798 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:35.241874 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:35.241918 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:35.242052 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:35.242299 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 02:48:35.242364 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:48:35.242620 10293 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260812 02:48:35.242992 10293 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:35.243184 10293 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:35.243399 10293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:35.246404 10293 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260812 02:48:35.247103 10293 leader_election.cc:302] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc5c5b5beb9f44a4a790f287aacfe9fd; no voters:
I20260812 02:48:35.247922 10293 leader_election.cc:288] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:35.248059 10307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:35.248876 10307 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [term 1 LEADER]: Becoming Leader. State: Replica: bc5c5b5beb9f44a4a790f287aacfe9fd, State: Running, Role: LEADER
I20260812 02:48:35.249677 10307 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260812 02:48:35.250576 10293 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:35.252976 10310 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc5c5b5beb9f44a4a790f287aacfe9fd. Latest consensus state: current_term: 1 leader_uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } } }
I20260812 02:48:35.253021 10309 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5c5b5beb9f44a4a790f287aacfe9fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } } }
I20260812 02:48:35.253511 10310 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:35.253523 10309 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:35.259373 10314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:35.265679 10314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:35.275949 10314 catalog_manager.cc:1347] Generated new cluster ID: 65c7124b8fb847ee9246b93b7a83d8a8
I20260812 02:48:35.276101 10314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:35.307144 10314 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:35.308836 10314 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:35.317436 10314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc5c5b5beb9f44a4a790f287aacfe9fd: Generated new TSK 0
I20260812 02:48:35.318471 10314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:35.328779 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10214
W20260812 02:48:35.330685 10284 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:35219
W20260812 02:48:35.331777 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10214: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:35.332001 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 10214
W20260812 02:48:35.356256 10196 connection.cc:535] client connection to 127.0.0.1:46875 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46875): Connection reset by peer (error 104)
2026-08-12T02:48:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:35 dist-test-slave-ss4n krb5kdc[10332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:35 dist-test-slave-ss4n krb5kdc[10332](info): set up 2 sockets
Aug 12 02:48:35 dist-test-slave-ss4n krb5kdc[10332](info): commencing operation
krb5kdc: starting...
W20260812 02:48:36.614809 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:36 dist-test-slave-ss4n krb5kdc[10332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:36.994313 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42455
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:37.384151 10347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:37.384583 10347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:37.384699 10347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:37.392489 10347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:37.392635 10347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:37.392683 10347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:37.392740 10347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:37.392786 10347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:37.403412 10347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:37.404781 10347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:37.406213 10347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:37.419427 10352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:37.419651 10355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:37.420090 10353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:37.420388 10347 server_base.cc:1034] running on GCE node
I20260812 02:48:37.421392 10347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:37.423195 10347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:37.424470 10347 hybrid_clock.cc:648] HybridClock initialized: now 1786502917424422 us; error 43 us; skew 500 ppm
I20260812 02:48:37.424904 10347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:37 dist-test-slave-ss4n krb5kdc[10332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:37.428957 10347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:37.431134 10347 webserver.cc:460] Webserver started at http://127.0.0.1:42465/ using document root <none> and password file <none>
I20260812 02:48:37.431951 10347 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:37.432122 10347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:37.432461 10347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:37.435804 10347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "5d5687c9d3ef41948c75a466b407e7d4"
format_stamp: "Formatted at 2026-08-12 02:48:37 on dist-test-slave-ss4n"
server_key: "e0ed4766f0654d9debd8369aeb826d9a"
server_key_iv: "acdaf92087afe1954d17fab9b81b1fae"
server_key_version: "encryptionkey@0"
I20260812 02:48:37.436666 10347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "5d5687c9d3ef41948c75a466b407e7d4"
format_stamp: "Formatted at 2026-08-12 02:48:37 on dist-test-slave-ss4n"
server_key: "e0ed4766f0654d9debd8369aeb826d9a"
server_key_iv: "acdaf92087afe1954d17fab9b81b1fae"
server_key_version: "encryptionkey@0"
I20260812 02:48:37.443324 10347 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.007s
I20260812 02:48:37.447530 10362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:37.449185 10347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:48:37.449431 10347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "5d5687c9d3ef41948c75a466b407e7d4"
format_stamp: "Formatted at 2026-08-12 02:48:37 on dist-test-slave-ss4n"
server_key: "e0ed4766f0654d9debd8369aeb826d9a"
server_key_iv: "acdaf92087afe1954d17fab9b81b1fae"
server_key_version: "encryptionkey@0"
I20260812 02:48:37.449736 10347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:37.456203 10347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:37.461431 10347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:37.461861 10347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:37.484192 10347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42455
I20260812 02:48:37.484256 10417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42455 every 8 connection(s)
I20260812 02:48:37.485962 10347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:37.490486 10419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:37.492125 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 10347
I20260812 02:48:37.492368 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:37.492885 18195 external_mini_cluster.cc:1428] Setting key cac76d4cda4f67b7c1f21cb0c1a847b0
I20260812 02:48:37.503450 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:37.495300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:40189 (local address 127.0.0.1:42455)
0812 02:48:37.496267 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:37.496290 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:37.496356 (+ 66us) server_negotiation.cc:407] Connection header received
0812 02:48:37.498535 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:37.498576 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:37.499200 (+ 624us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:37.499785 (+ 585us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:37.501037 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:37.501056 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:37.501078 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:37.501099 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:37.501190 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:37.501228 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:37.501527 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:37.501684 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 02:48:37.501933 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":393,"threads_started":1}
I20260812 02:48:37.508092 10419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4: Bootstrap starting.
I20260812 02:48:37.508435 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:37.512121 10419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:37.513396 10419 log.cc:824] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:37.516958 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:37.513847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:45419 (local address 127.0.0.1:42455)
0812 02:48:37.514123 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:37.514134 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:37.514634 (+ 500us) server_negotiation.cc:407] Connection header received
0812 02:48:37.514886 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:37.514900 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:37.514960 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:37.515079 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:37.516160 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:37.516167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:37.516172 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:37.516205 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:37.516251 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:37.516268 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:37.516410 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:37.516530 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 02:48:37.516675 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:48:37.517518 10419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4: No bootstrap required, opened a new log
I20260812 02:48:37.526413 10419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5687c9d3ef41948c75a466b407e7d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42455 } }
I20260812 02:48:37.526907 10419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:37.527055 10419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5687c9d3ef41948c75a466b407e7d4, State: Initialized, Role: FOLLOWER
I20260812 02:48:37.527746 10419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5687c9d3ef41948c75a466b407e7d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42455 } }
I20260812 02:48:37.528020 10419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:37.528195 10419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:37.528402 10419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:37.531327 10419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5687c9d3ef41948c75a466b407e7d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42455 } }
I20260812 02:48:37.531917 10419 leader_election.cc:302] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d5687c9d3ef41948c75a466b407e7d4; no voters:
I20260812 02:48:37.532632 10419 leader_election.cc:288] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:37.532830 10431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:37.533704 10431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 5d5687c9d3ef41948c75a466b407e7d4, State: Running, Role: LEADER
I20260812 02:48:37.534309 10431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5687c9d3ef41948c75a466b407e7d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42455 } }
I20260812 02:48:37.535144 10419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:37.537925 10432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d5687c9d3ef41948c75a466b407e7d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5687c9d3ef41948c75a466b407e7d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42455 } } }
I20260812 02:48:37.537850 10433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d5687c9d3ef41948c75a466b407e7d4. Latest consensus state: current_term: 1 leader_uuid: "5d5687c9d3ef41948c75a466b407e7d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5687c9d3ef41948c75a466b407e7d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42455 } } }
I20260812 02:48:37.538503 10432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:37.538515 10433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:37.543484 10442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:37.550143 10442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:37.561570 10442 catalog_manager.cc:1347] Generated new cluster ID: 2c8dbad6c6f2443a8f6b36147f475134
I20260812 02:48:37.561719 10442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:37.587116 10442 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:37.587692 10442 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:37.595063 10442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d5687c9d3ef41948c75a466b407e7d4: Generated new TSK 0
I20260812 02:48:37.596264 10442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:37.608163 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10347
W20260812 02:48:37.609572 10413 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:40189
W20260812 02:48:37.610520 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10347: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:37.610723 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 10347
2026-08-12T02:48:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:37 dist-test-slave-ss4n krb5kdc[10461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:37 dist-test-slave-ss4n krb5kdc[10461](info): set up 2 sockets
Aug 12 02:48:37 dist-test-slave-ss4n krb5kdc[10461](info): commencing operation
krb5kdc: starting...
W20260812 02:48:38.924185 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:39 dist-test-slave-ss4n krb5kdc[10461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:39.293967 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41175
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:39.663697 10478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:39.664031 10478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:39.664217 10478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:39.672041 10478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:39.672190 10478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:39.672310 10478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:39.672413 10478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:39.672466 10478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:39.683317 10478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41175
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:39.684880 10478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:39.686359 10478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:39.701216 10488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:39.701164 10483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:39.701254 10484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:39.701555 10478 server_base.cc:1034] running on GCE node
I20260812 02:48:39.702736 10478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:39.704308 10478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:39.705674 10478 hybrid_clock.cc:648] HybridClock initialized: now 1786502919705604 us; error 53 us; skew 500 ppm
I20260812 02:48:39.706131 10478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:39 dist-test-slave-ss4n krb5kdc[10461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:39.709976 10478 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:39.711867 10478 webserver.cc:460] Webserver started at http://127.0.0.1:44715/ using document root <none> and password file <none>
I20260812 02:48:39.712679 10478 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:39.712841 10478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:39.713174 10478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:39.716478 10478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "d744f33f146a4d3c9159873dfc0beca8"
format_stamp: "Formatted at 2026-08-12 02:48:39 on dist-test-slave-ss4n"
server_key: "3808f6946dfec5fb0402e26c5b723c85"
server_key_iv: "e8444bb7c768a0860b42a2316f97eca6"
server_key_version: "encryptionkey@0"
I20260812 02:48:39.717387 10478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "d744f33f146a4d3c9159873dfc0beca8"
format_stamp: "Formatted at 2026-08-12 02:48:39 on dist-test-slave-ss4n"
server_key: "3808f6946dfec5fb0402e26c5b723c85"
server_key_iv: "e8444bb7c768a0860b42a2316f97eca6"
server_key_version: "encryptionkey@0"
I20260812 02:48:39.723250 10478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 02:48:39.727229 10496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:39.728791 10478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:48:39.729135 10478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "d744f33f146a4d3c9159873dfc0beca8"
format_stamp: "Formatted at 2026-08-12 02:48:39 on dist-test-slave-ss4n"
server_key: "3808f6946dfec5fb0402e26c5b723c85"
server_key_iv: "e8444bb7c768a0860b42a2316f97eca6"
server_key_version: "encryptionkey@0"
I20260812 02:48:39.729368 10478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:39.748183 10478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:39.753248 10478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:39.753610 10478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:39.773463 10478 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41175
I20260812 02:48:39.773509 10563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41175 every 8 connection(s)
I20260812 02:48:39.774933 10478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:39.778863 10565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:39.782099 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 10478
I20260812 02:48:39.782310 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:39.782869 18195 external_mini_cluster.cc:1428] Setting key 1222dcbe47d4efd12e28c846715816af
I20260812 02:48:39.791527 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8: Bootstrap starting.
I20260812 02:48:39.794934 10565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:39.796378 10565 log.cc:824] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:48:39 dist-test-slave-ss4n krb5kdc[10461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 02:48:39.799834 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8: No bootstrap required, opened a new log
I20260812 02:48:39.809687 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:39.784892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:42581 (local address 127.0.0.1:41175)
0812 02:48:39.785842 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:39.785854 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:39.785897 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:48:39.787790 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:39.787821 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:39.788263 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:39.788699 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:39.790429 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:39.791898 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:39.793361 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:39.794198 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:39.794404 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:39.799621 (+ 5217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:39.799633 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:39.799655 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:39.799702 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:39.804042 (+ 4340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:39.805126 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:39.805136 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:39.805145 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:39.805371 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:39.806018 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:39.806023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:39.806026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:39.806817 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:39.807173 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:39.807694 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 02:48:39.808106 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":384,"threads_started":1}
I20260812 02:48:39.811576 10565 raft_consensus.cc:348] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d744f33f146a4d3c9159873dfc0beca8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } }
I20260812 02:48:39.812026 10565 raft_consensus.cc:374] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:39.812157 10565 raft_consensus.cc:729] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d744f33f146a4d3c9159873dfc0beca8, State: Initialized, Role: FOLLOWER
I20260812 02:48:39.812816 10565 consensus_queue.cc:260] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d744f33f146a4d3c9159873dfc0beca8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } }
I20260812 02:48:39.813107 10565 raft_consensus.cc:388] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:39.813275 10565 raft_consensus.cc:482] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:39.813462 10565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:39.814850 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:39.816278 10565 raft_consensus.cc:504] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d744f33f146a4d3c9159873dfc0beca8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } }
I20260812 02:48:39.816845 10565 leader_election.cc:302] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d744f33f146a4d3c9159873dfc0beca8; no voters:
I20260812 02:48:39.817588 10565 leader_election.cc:288] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:39.817775 10571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:39.819515 10571 raft_consensus.cc:686] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [term 1 LEADER]: Becoming Leader. State: Replica: d744f33f146a4d3c9159873dfc0beca8, State: Running, Role: LEADER
I20260812 02:48:39.820214 10571 consensus_queue.cc:237] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d744f33f146a4d3c9159873dfc0beca8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } }
I20260812 02:48:39.820384 10565 sys_catalog.cc:564] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:39.823622 10574 sys_catalog.cc:455] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d744f33f146a4d3c9159873dfc0beca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d744f33f146a4d3c9159873dfc0beca8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } } }
I20260812 02:48:39.824144 10574 sys_catalog.cc:458] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:39.823593 10576 sys_catalog.cc:455] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d744f33f146a4d3c9159873dfc0beca8. Latest consensus state: current_term: 1 leader_uuid: "d744f33f146a4d3c9159873dfc0beca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d744f33f146a4d3c9159873dfc0beca8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } } }
I20260812 02:48:39.825223 10576 sys_catalog.cc:458] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:39.826470 10582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:39.833616 10582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:39.841197 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:39.820716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:47725 (local address 127.0.0.1:41175)
0812 02:48:39.820967 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:39.820978 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:39.821585 (+ 607us) server_negotiation.cc:407] Connection header received
0812 02:48:39.821817 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:39.821822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:39.821927 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:39.822065 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:39.824460 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:39.825829 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:39.827431 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:39.828217 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:39.828370 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:39.832031 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:39.832040 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:39.832045 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:39.832091 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:39.836464 (+ 4373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:39.837748 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:39.837755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:39.837759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:39.837956 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:39.838602 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:39.838608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:39.838611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:39.839071 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:39.839369 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:39.840594 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0812 02:48:39.840805 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:48:39.847273 10582 catalog_manager.cc:1347] Generated new cluster ID: e8fb020eee964179b4969c5d6cb05193
I20260812 02:48:39.847406 10582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:39.886078 10582 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:39.887583 10582 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:39.894459 10582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d744f33f146a4d3c9159873dfc0beca8: Generated new TSK 0
I20260812 02:48:39.895541 10582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:39.902173 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10478
I20260812 02:48:39.903478 10559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:40.003510 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 10478
W20260812 02:48:40.025952 10459 connection.cc:535] client connection to 127.0.0.1:41175 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41175): Connection reset by peer (error 104)
2026-08-12T02:48:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2388 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:40 dist-test-slave-ss4n krb5kdc[10605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:40 dist-test-slave-ss4n krb5kdc[10605](info): set up 2 sockets
Aug 12 02:48:40 dist-test-slave-ss4n krb5kdc[10605](info): commencing operation
krb5kdc: starting...
W20260812 02:48:41.329833 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:41 dist-test-slave-ss4n krb5kdc[10605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:41.711155 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34601
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:42.094602 10621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:42.094957 10621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:42.095057 10621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:42.102664 10621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:42.102794 10621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:42.102840 10621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:42.102895 10621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:42.102941 10621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:42.113708 10621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34601
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:42.115074 10621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:42.116425 10621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:42.129819 10626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:42.130244 10627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:42.130324 10630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:42.131398 10621 server_base.cc:1034] running on GCE node
I20260812 02:48:42.132333 10621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:42.134265 10621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:42.135797 10621 hybrid_clock.cc:648] HybridClock initialized: now 1786502922135735 us; error 57 us; skew 500 ppm
I20260812 02:48:42.136267 10621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:42 dist-test-slave-ss4n krb5kdc[10605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:42.145287 10621 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:42.147234 10621 webserver.cc:460] Webserver started at http://127.0.0.1:45921/ using document root <none> and password file <none>
I20260812 02:48:42.148097 10621 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:42.148300 10621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:42.148663 10621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:42.152436 10621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "69698b36cc0d44fdba49ea29bfe08986"
format_stamp: "Formatted at 2026-08-12 02:48:42 on dist-test-slave-ss4n"
server_key: "6b94e6fd34fc1a3e0e45d96666258f56"
server_key_iv: "904dab191b285ab8cc6b6232854d0224"
server_key_version: "encryptionkey@0"
I20260812 02:48:42.153455 10621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "69698b36cc0d44fdba49ea29bfe08986"
format_stamp: "Formatted at 2026-08-12 02:48:42 on dist-test-slave-ss4n"
server_key: "6b94e6fd34fc1a3e0e45d96666258f56"
server_key_iv: "904dab191b285ab8cc6b6232854d0224"
server_key_version: "encryptionkey@0"
I20260812 02:48:42.159910 10621 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:48:42.164219 10637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:42.166090 10621 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:48:42.166307 10621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "69698b36cc0d44fdba49ea29bfe08986"
format_stamp: "Formatted at 2026-08-12 02:48:42 on dist-test-slave-ss4n"
server_key: "6b94e6fd34fc1a3e0e45d96666258f56"
server_key_iv: "904dab191b285ab8cc6b6232854d0224"
server_key_version: "encryptionkey@0"
I20260812 02:48:42.166549 10621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:42.188144 10621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:42.194265 10621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:42.194609 10621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:42.218393 10621 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34601
I20260812 02:48:42.218402 10700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34601 every 8 connection(s)
I20260812 02:48:42.220180 10621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:42.220616 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 10621
I20260812 02:48:42.220810 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:42.221396 18195 external_mini_cluster.cc:1428] Setting key 41beccd71ed63014246ff34c4c0fa57c
I20260812 02:48:42.225878 10701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:42.236738 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:42.223644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:47607 (local address 127.0.0.1:34601)
0812 02:48:42.224548 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:42.224563 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:42.224611 (+ 48us) server_negotiation.cc:407] Connection header received
0812 02:48:42.226629 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:42.226656 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:42.227104 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:42.227547 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:42.228146 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:42.229673 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:42.231199 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:42.232032 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:42.232268 (+ 236us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:42.233992 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:42.234002 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:42.234021 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:42.234701 (+ 680us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:42.234808 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:42.234849 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:42.235075 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:42.235431 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 02:48:42.235638 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":435,"threads_started":1}
I20260812 02:48:42.239516 10701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986: Bootstrap starting.
I20260812 02:48:42.241703 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:42.243475 10701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:42.244740 10701 log.cc:824] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:42.248776 10701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986: No bootstrap required, opened a new log
I20260812 02:48:42.254912 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:42.247102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:54591 (local address 127.0.0.1:34601)
0812 02:48:42.247300 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:42.247307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:42.247742 (+ 435us) server_negotiation.cc:407] Connection header received
0812 02:48:42.247946 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:42.247952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:42.248044 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:42.248191 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:42.248777 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:42.250090 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:42.251568 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:42.252414 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:42.252569 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:42.253906 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:42.253912 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:42.253917 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:42.253955 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:42.254007 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:42.254029 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:42.254156 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:42.254413 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 02:48:42.254486 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:48:42.259575 10701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69698b36cc0d44fdba49ea29bfe08986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } }
I20260812 02:48:42.260026 10701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:42.260141 10701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69698b36cc0d44fdba49ea29bfe08986, State: Initialized, Role: FOLLOWER
I20260812 02:48:42.260869 10701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69698b36cc0d44fdba49ea29bfe08986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } }
I20260812 02:48:42.261140 10701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:42.261313 10701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:42.261627 10701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:42.264839 10701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69698b36cc0d44fdba49ea29bfe08986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } }
I20260812 02:48:42.265429 10701 leader_election.cc:302] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69698b36cc0d44fdba49ea29bfe08986; no voters:
I20260812 02:48:42.266208 10701 leader_election.cc:288] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:42.266371 10712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:42.267637 10712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [term 1 LEADER]: Becoming Leader. State: Replica: 69698b36cc0d44fdba49ea29bfe08986, State: Running, Role: LEADER
I20260812 02:48:42.268225 10712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69698b36cc0d44fdba49ea29bfe08986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } }
I20260812 02:48:42.268688 10701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:42.271368 10714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69698b36cc0d44fdba49ea29bfe08986. Latest consensus state: current_term: 1 leader_uuid: "69698b36cc0d44fdba49ea29bfe08986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69698b36cc0d44fdba49ea29bfe08986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } } }
I20260812 02:48:42.271858 10714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:42.271613 10713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69698b36cc0d44fdba49ea29bfe08986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69698b36cc0d44fdba49ea29bfe08986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } } }
I20260812 02:48:42.272687 10713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:42.281327 10723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:42.286495 10723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:42.296151 10723 catalog_manager.cc:1347] Generated new cluster ID: 39d566dec9cd497f8d7707a9876f725b
I20260812 02:48:42.296303 10723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:42.314663 10723 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:42.316198 10723 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:42.328272 10723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69698b36cc0d44fdba49ea29bfe08986: Generated new TSK 0
I20260812 02:48:42.329303 10723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:42.337699 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10621
W20260812 02:48:42.339069 10696 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:47607
W20260812 02:48:42.339929 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10621: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:42.340090 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 10621
W20260812 02:48:42.362499 10603 connection.cc:535] client connection to 127.0.0.1:34601 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34601): Connection reset by peer (error 104)
2026-08-12T02:48:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:42 dist-test-slave-ss4n krb5kdc[10739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:42 dist-test-slave-ss4n krb5kdc[10739](info): set up 2 sockets
Aug 12 02:48:42 dist-test-slave-ss4n krb5kdc[10739](info): commencing operation
krb5kdc: starting...
W20260812 02:48:43.682858 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:43 dist-test-slave-ss4n krb5kdc[10739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:48:44.070165 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33623
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:44.438334 10755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:44.438681 10755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:44.438870 10755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:44.446558 10755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:44.446695 10755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:44.446802 10755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:44.446899 10755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:44.447005 10755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:44.457739 10755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:44.459264 10755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:44.460745 10755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:44.474105 10760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:44.474265 10764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:44.474195 10761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:44.475693 10755 server_base.cc:1034] running on GCE node
I20260812 02:48:44.476964 10755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:44.478766 10755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:44.480062 10755 hybrid_clock.cc:648] HybridClock initialized: now 1786502924480021 us; error 67 us; skew 500 ppm
I20260812 02:48:44.480499 10755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:44 dist-test-slave-ss4n krb5kdc[10739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502924, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:44.484233 10755 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:48:44.485967 10755 webserver.cc:460] Webserver started at http://127.0.0.1:46595/ using document root <none> and password file <none>
I20260812 02:48:44.486711 10755 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:44.486872 10755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:44.487211 10755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:44.490412 10755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "c7956095cb084d2faf8b77d1ba307993"
format_stamp: "Formatted at 2026-08-12 02:48:44 on dist-test-slave-ss4n"
server_key: "84d4ac1f7af25591fba9f3e974a98976"
server_key_iv: "1d8d5f6a7a36a100404d9cac5baa4b68"
server_key_version: "encryptionkey@0"
I20260812 02:48:44.491247 10755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "c7956095cb084d2faf8b77d1ba307993"
format_stamp: "Formatted at 2026-08-12 02:48:44 on dist-test-slave-ss4n"
server_key: "84d4ac1f7af25591fba9f3e974a98976"
server_key_iv: "1d8d5f6a7a36a100404d9cac5baa4b68"
server_key_version: "encryptionkey@0"
I20260812 02:48:44.497054 10755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:48:44.501232 10773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:44.503077 10755 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260812 02:48:44.503422 10755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "c7956095cb084d2faf8b77d1ba307993"
format_stamp: "Formatted at 2026-08-12 02:48:44 on dist-test-slave-ss4n"
server_key: "84d4ac1f7af25591fba9f3e974a98976"
server_key_iv: "1d8d5f6a7a36a100404d9cac5baa4b68"
server_key_version: "encryptionkey@0"
I20260812 02:48:44.503691 10755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:44.510773 10755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:44.516113 10755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:44.516477 10755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:44.538216 10755 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33623
I20260812 02:48:44.538272 10832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33623 every 8 connection(s)
I20260812 02:48:44.539866 10755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:44.544162 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:44.546248 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 10755
I20260812 02:48:44.546455 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:44.547049 18195 external_mini_cluster.cc:1428] Setting key aefe863550d87fbbd183d9c35e83a35c
I20260812 02:48:44.555994 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:44.549181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:52201 (local address 127.0.0.1:33623)
0812 02:48:44.550093 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:44.550106 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:44.550190 (+ 84us) server_negotiation.cc:407] Connection header received
0812 02:48:44.551903 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:44.551932 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:44.552395 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:44.552892 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:44.554228 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:44.554235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:44.554250 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:44.554265 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:44.554325 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:44.554350 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:44.554633 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:44.554776 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 02:48:44.554987 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":403,"threads_started":1}
I20260812 02:48:44.560770 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:44.560925 10835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993: Bootstrap starting.
I20260812 02:48:44.564621 10835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:44.566119 10835 log.cc:824] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:44.568751 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:44.565666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:55813 (local address 127.0.0.1:33623)
0812 02:48:44.565941 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:44.565949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:44.566407 (+ 458us) server_negotiation.cc:407] Connection header received
0812 02:48:44.566576 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:44.566583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:44.566661 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:44.566833 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:44.568048 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:44.568056 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:44.568061 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:44.568080 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:44.568124 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:44.568145 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:44.568273 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:44.568390 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 02:48:44.568453 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:48:44.570386 10835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993: No bootstrap required, opened a new log
I20260812 02:48:44.578473 10835 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7956095cb084d2faf8b77d1ba307993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33623 } }
I20260812 02:48:44.578929 10835 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:44.579041 10835 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7956095cb084d2faf8b77d1ba307993, State: Initialized, Role: FOLLOWER
I20260812 02:48:44.579726 10835 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7956095cb084d2faf8b77d1ba307993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33623 } }
I20260812 02:48:44.580053 10835 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:44.580217 10835 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:44.580410 10835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:44.582885 10835 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7956095cb084d2faf8b77d1ba307993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33623 } }
I20260812 02:48:44.583439 10835 leader_election.cc:302] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7956095cb084d2faf8b77d1ba307993; no voters:
I20260812 02:48:44.584059 10835 leader_election.cc:288] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:44.584271 10845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:44.585028 10845 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [term 1 LEADER]: Becoming Leader. State: Replica: c7956095cb084d2faf8b77d1ba307993, State: Running, Role: LEADER
I20260812 02:48:44.585691 10845 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7956095cb084d2faf8b77d1ba307993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33623 } }
I20260812 02:48:44.586328 10835 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:44.589509 10848 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7956095cb084d2faf8b77d1ba307993. Latest consensus state: current_term: 1 leader_uuid: "c7956095cb084d2faf8b77d1ba307993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7956095cb084d2faf8b77d1ba307993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33623 } } }
I20260812 02:48:44.589460 10847 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7956095cb084d2faf8b77d1ba307993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7956095cb084d2faf8b77d1ba307993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33623 } } }
I20260812 02:48:44.590078 10848 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:44.590086 10847 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:44.592597 10855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:44.599601 10855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:44.610848 10855 catalog_manager.cc:1347] Generated new cluster ID: ae3c9c0ebd24447f9cc6ac61ec87c30b
I20260812 02:48:44.610975 10855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:44.623531 10855 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:44.624111 10855 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:44.635094 10855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7956095cb084d2faf8b77d1ba307993: Generated new TSK 0
I20260812 02:48:44.636483 10855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:44.652971 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10755
W20260812 02:48:44.654335 10828 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:52201
W20260812 02:48:44.655195 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10755: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:44.655365 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 10755
2026-08-12T02:48:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:44 dist-test-slave-ss4n krb5kdc[10871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:44 dist-test-slave-ss4n krb5kdc[10871](info): set up 2 sockets
Aug 12 02:48:44 dist-test-slave-ss4n krb5kdc[10871](info): commencing operation
krb5kdc: starting...
W20260812 02:48:45.960940 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:46 dist-test-slave-ss4n krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:48:46.347926 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:43885
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:43885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:46.736971 10886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:46.737315 10886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:46.737547 10886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:46.745332 10886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:46.745471 10886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:46.745618 10886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:46.745719 10886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:46.745787 10886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:46.757161 10886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:43885
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:43885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:46.758746 10886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:46.760180 10886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:46.773820 10894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:46.773854 10892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:46.773775 10891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:46.774997 10886 server_base.cc:1034] running on GCE node
I20260812 02:48:46.775940 10886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:46.777841 10886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:46.779121 10886 hybrid_clock.cc:648] HybridClock initialized: now 1786502926779037 us; error 51 us; skew 500 ppm
I20260812 02:48:46.779515 10886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:46 dist-test-slave-ss4n krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502926, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:46.783380 10886 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:48:46.785336 10886 webserver.cc:460] Webserver started at http://127.17.196.254:45455/ using document root <none> and password file <none>
I20260812 02:48:46.786188 10886 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:46.786310 10886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:46.786594 10886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:46.789949 10886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "1b0981dc044644b79e3fdb6009b81f70"
format_stamp: "Formatted at 2026-08-12 02:48:46 on dist-test-slave-ss4n"
server_key: "28a9f21b1a55c805c8f287c1c99fcfea"
server_key_iv: "3b184d4859b235e126c6e70a2dc0d9b8"
server_key_version: "encryptionkey@0"
I20260812 02:48:46.790947 10886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "1b0981dc044644b79e3fdb6009b81f70"
format_stamp: "Formatted at 2026-08-12 02:48:46 on dist-test-slave-ss4n"
server_key: "28a9f21b1a55c805c8f287c1c99fcfea"
server_key_iv: "3b184d4859b235e126c6e70a2dc0d9b8"
server_key_version: "encryptionkey@0"
I20260812 02:48:46.798151 10886 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260812 02:48:46.802785 10902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:46.804711 10886 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:48:46.804988 10886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "1b0981dc044644b79e3fdb6009b81f70"
format_stamp: "Formatted at 2026-08-12 02:48:46 on dist-test-slave-ss4n"
server_key: "28a9f21b1a55c805c8f287c1c99fcfea"
server_key_iv: "3b184d4859b235e126c6e70a2dc0d9b8"
server_key_version: "encryptionkey@0"
I20260812 02:48:46.805233 10886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:46.823956 10886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:46.829352 10886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:46.829759 10886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:46.851954 10886 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:43885
I20260812 02:48:46.852010 10964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:43885 every 8 connection(s)
I20260812 02:48:46.854019 10886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:46.857299 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 10886
I20260812 02:48:46.857502 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:46.858083 18195 external_mini_cluster.cc:1428] Setting key 0283d831307fe22fe2d8adebe3b5e5c0
I20260812 02:48:46.858677 10965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:48:46 dist-test-slave-ss4n krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:46.873171 10965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70: Bootstrap starting.
I20260812 02:48:46.877298 10965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:46.878625 10965 log.cc:824] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:46.881918 10965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70: No bootstrap required, opened a new log
I20260812 02:48:46.884089 10968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:46.860112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38051 (local address 127.17.196.254:43885)
0812 02:48:46.861045 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:46.861057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:46.861104 (+ 47us) server_negotiation.cc:407] Connection header received
0812 02:48:46.862914 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:46.862946 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:46.863418 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:46.863902 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:46.865614 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:46.866998 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:46.868374 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:46.869111 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:46.869346 (+ 235us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:46.874735 (+ 5389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:46.874747 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:46.874762 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:46.874809 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:46.878778 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:46.879902 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:46.879910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:46.879918 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:46.880130 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:46.880749 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:46.880761 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:46.880763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:46.881388 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:46.881855 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:46.882473 (+ 618us) server_negotiation.cc:299] Negotiation successful
0812 02:48:46.882866 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":434,"threads_started":1}
I20260812 02:48:46.889389 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:46.891594 10965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0981dc044644b79e3fdb6009b81f70" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43885 } }
I20260812 02:48:46.892019 10965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:46.892155 10965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0981dc044644b79e3fdb6009b81f70, State: Initialized, Role: FOLLOWER
I20260812 02:48:46.892833 10965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0981dc044644b79e3fdb6009b81f70" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43885 } }
I20260812 02:48:46.893095 10965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:46.893256 10965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:46.893471 10965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:46.896963 10965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0981dc044644b79e3fdb6009b81f70" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43885 } }
I20260812 02:48:46.897862 10965 leader_election.cc:302] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b0981dc044644b79e3fdb6009b81f70; no voters:
I20260812 02:48:46.898729 10965 leader_election.cc:288] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:46.898936 10976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:46.899793 10976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [term 1 LEADER]: Becoming Leader. State: Replica: 1b0981dc044644b79e3fdb6009b81f70, State: Running, Role: LEADER
I20260812 02:48:46.901154 10976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0981dc044644b79e3fdb6009b81f70" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43885 } }
I20260812 02:48:46.902416 10965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:46.904896 10978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b0981dc044644b79e3fdb6009b81f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0981dc044644b79e3fdb6009b81f70" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43885 } } }
I20260812 02:48:46.905462 10978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:46.911027 10977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b0981dc044644b79e3fdb6009b81f70. Latest consensus state: current_term: 1 leader_uuid: "1b0981dc044644b79e3fdb6009b81f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0981dc044644b79e3fdb6009b81f70" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 43885 } } }
I20260812 02:48:46.912251 10977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:46.917224 10984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:46.917811 10968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:46.894444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48151 (local address 127.17.196.254:43885)
0812 02:48:46.894647 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:46.894654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:46.895186 (+ 532us) server_negotiation.cc:407] Connection header received
0812 02:48:46.895304 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:46.895310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:46.895414 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:46.895549 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:46.897126 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:46.898488 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:46.900835 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:46.901653 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:46.901833 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:46.905596 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:46.905603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:46.905608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:46.905660 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:46.911170 (+ 5510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:46.913088 (+ 1918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:46.913099 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:46.913106 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:46.913469 (+ 363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:46.914524 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:46.914534 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:46.914545 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:46.915216 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:46.915868 (+ 652us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:46.916963 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0812 02:48:46.917295 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:48:46.924777 10984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:46.935786 10984 catalog_manager.cc:1347] Generated new cluster ID: c5a9e61872584917b898932ea7ddc797
I20260812 02:48:46.935945 10984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:46.948854 10984 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:46.950857 10984 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:46.962966 10984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b0981dc044644b79e3fdb6009b81f70: Generated new TSK 0
I20260812 02:48:46.964011 10984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:46.977062 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 10886
I20260812 02:48:46.979189 10959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:47.077265 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 10886
W20260812 02:48:47.101918 10869 connection.cc:535] client connection to 127.17.196.254:43885 recv error: Network error: failed to read from TLS socket (remote: 127.17.196.254:43885): Connection reset by peer (error 104)
2026-08-12T02:48:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:47 dist-test-slave-ss4n krb5kdc[11004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:47 dist-test-slave-ss4n krb5kdc[11004](info): set up 2 sockets
Aug 12 02:48:47 dist-test-slave-ss4n krb5kdc[11004](info): commencing operation
krb5kdc: starting...
W20260812 02:48:48.360437 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.003s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:48 dist-test-slave-ss4n krb5kdc[11004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:48:48.738848 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:35241
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:35241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:49.118352 11021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:49.118708 11021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:49.118925 11021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:49.126819 11021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:49.126950 11021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:49.127060 11021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:49.127200 11021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:49.127265 11021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:49.138123 11021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:35241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:35241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:49.139573 11021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:49.141052 11021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:49.154497 11030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:49.154646 11026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:49.155221 11027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:49.155958 11021 server_base.cc:1034] running on GCE node
I20260812 02:48:49.156876 11021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:49.158496 11021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:49.159726 11021 hybrid_clock.cc:648] HybridClock initialized: now 1786502929159678 us; error 37 us; skew 500 ppm
I20260812 02:48:49.160110 11021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:49 dist-test-slave-ss4n krb5kdc[11004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:49.163740 11021 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:48:49.165578 11021 webserver.cc:460] Webserver started at http://127.17.196.254:35775/ using document root <none> and password file <none>
I20260812 02:48:49.166301 11021 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:49.166411 11021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:49.166695 11021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:49.169888 11021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "98b5fc4062e148a3ad22ade7870f4af1"
format_stamp: "Formatted at 2026-08-12 02:48:49 on dist-test-slave-ss4n"
server_key: "c73cca69f79a40a4f1e60f8f4e5685ab"
server_key_iv: "1c6497463c28e8a7615fe7f61566c446"
server_key_version: "encryptionkey@0"
I20260812 02:48:49.170661 11021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "98b5fc4062e148a3ad22ade7870f4af1"
format_stamp: "Formatted at 2026-08-12 02:48:49 on dist-test-slave-ss4n"
server_key: "c73cca69f79a40a4f1e60f8f4e5685ab"
server_key_iv: "1c6497463c28e8a7615fe7f61566c446"
server_key_version: "encryptionkey@0"
I20260812 02:48:49.176184 11021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:48:49.180267 11038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:49.182065 11021 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 02:48:49.182291 11021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "98b5fc4062e148a3ad22ade7870f4af1"
format_stamp: "Formatted at 2026-08-12 02:48:49 on dist-test-slave-ss4n"
server_key: "c73cca69f79a40a4f1e60f8f4e5685ab"
server_key_iv: "1c6497463c28e8a7615fe7f61566c446"
server_key_version: "encryptionkey@0"
I20260812 02:48:49.182528 11021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:49.200345 11021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:49.205444 11021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:49.205816 11021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:49.227244 11021 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:35241
I20260812 02:48:49.227277 11100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:35241 every 8 connection(s)
I20260812 02:48:49.228847 11021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:49.233228 11101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:49.238015 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 11021
I20260812 02:48:49.238230 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:49.238682 18195 external_mini_cluster.cc:1428] Setting key ed16e043ddb06a8edbcc25a5647caf81
I20260812 02:48:49.245965 11101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1: Bootstrap starting.
I20260812 02:48:49.249878 11101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:49.251374 11101 log.cc:824] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:49.255110 11101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1: No bootstrap required, opened a new log
I20260812 02:48:49.255841 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:49.240838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53041 (local address 127.17.196.254:35241)
0812 02:48:49.241718 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:49.241731 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:49.241786 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:48:49.244465 (+ 2679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:49.244502 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:49.245039 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:49.245633 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:49.246357 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:49.248002 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:49.250130 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:49.251261 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:49.251470 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:49.252925 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:49.252934 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:49.252951 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:49.253662 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:49.253746 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:49.253773 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:49.253905 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:49.254359 (+ 454us) server_negotiation.cc:299] Negotiation successful
0812 02:48:49.254652 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":396,"threads_started":1}
I20260812 02:48:49.260735 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:49.264285 11101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b5fc4062e148a3ad22ade7870f4af1" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35241 } }
I20260812 02:48:49.264815 11101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:49.264953 11101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98b5fc4062e148a3ad22ade7870f4af1, State: Initialized, Role: FOLLOWER
I20260812 02:48:49.265722 11101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b5fc4062e148a3ad22ade7870f4af1" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35241 } }
I20260812 02:48:49.266119 11101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:49.266290 11101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:49.266497 11101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:49.269168 11101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b5fc4062e148a3ad22ade7870f4af1" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35241 } }
I20260812 02:48:49.269848 11101 leader_election.cc:302] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98b5fc4062e148a3ad22ade7870f4af1; no voters:
I20260812 02:48:49.270953 11101 leader_election.cc:288] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:49.271135 11112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:49.271973 11112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [term 1 LEADER]: Becoming Leader. State: Replica: 98b5fc4062e148a3ad22ade7870f4af1, State: Running, Role: LEADER
I20260812 02:48:49.272706 11112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b5fc4062e148a3ad22ade7870f4af1" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35241 } }
I20260812 02:48:49.273531 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:49.265726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59197 (local address 127.17.196.254:35241)
0812 02:48:49.265964 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:49.265973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:49.266438 (+ 465us) server_negotiation.cc:407] Connection header received
0812 02:48:49.266683 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:49.266693 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:49.266762 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:49.266891 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:49.267694 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:49.268544 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:49.270147 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:49.270735 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:49.270916 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:49.272361 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:49.272367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:49.272372 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:49.272397 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:49.272475 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:49.272496 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:49.272623 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:49.273079 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 02:48:49.273145 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:48:49.273785 11101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:49.276145 11114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98b5fc4062e148a3ad22ade7870f4af1. Latest consensus state: current_term: 1 leader_uuid: "98b5fc4062e148a3ad22ade7870f4af1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b5fc4062e148a3ad22ade7870f4af1" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35241 } } }
I20260812 02:48:49.276515 11113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98b5fc4062e148a3ad22ade7870f4af1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98b5fc4062e148a3ad22ade7870f4af1" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 35241 } } }
I20260812 02:48:49.276731 11114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:49.276811 11113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:49.278229 11120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:49.284189 11120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:49.295100 11120 catalog_manager.cc:1347] Generated new cluster ID: 1d81ab0174cc4c6aaf7a0a57e36baf58
I20260812 02:48:49.295334 11120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:49.312234 11120 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:49.313856 11120 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:49.328948 11120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98b5fc4062e148a3ad22ade7870f4af1: Generated new TSK 0
I20260812 02:48:49.330014 11120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:49.337548 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11021
W20260812 02:48:49.339030 11095 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53041
W20260812 02:48:49.339943 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11021: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:49.340133 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 11021
W20260812 02:48:49.364475 11001 connection.cc:535] client connection to 127.17.196.254:35241 recv error: Network error: failed to read from TLS socket (remote: 127.17.196.254:35241): Connection reset by peer (error 104)
2026-08-12T02:48:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2262 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:49 dist-test-slave-ss4n krb5kdc[11138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:49 dist-test-slave-ss4n krb5kdc[11138](info): set up 2 sockets
Aug 12 02:48:49 dist-test-slave-ss4n krb5kdc[11138](info): commencing operation
krb5kdc: starting...
W20260812 02:48:50.643827 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:50 dist-test-slave-ss4n krb5kdc[11138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:48:51.022439 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:45951
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:51.394037 11155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:51.394436 11155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:51.394627 11155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:51.405885 11155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:51.406003 11155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:51.406116 11155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:51.406219 11155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:51.406304 11155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:51.417886 11155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:45951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:51.419312 11155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:51.420725 11155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:51.433687 11163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:51.433738 11168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:51.433645 11164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:51.435099 11155 server_base.cc:1034] running on GCE node
I20260812 02:48:51.436175 11155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:51.437906 11155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:51.439198 11155 hybrid_clock.cc:648] HybridClock initialized: now 1786502931439143 us; error 51 us; skew 500 ppm
I20260812 02:48:51.439602 11155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:51 dist-test-slave-ss4n krb5kdc[11138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502931, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:51.443207 11155 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:48:51.444949 11155 webserver.cc:460] Webserver started at http://127.17.196.254:41061/ using document root <none> and password file <none>
I20260812 02:48:51.445870 11155 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:51.446046 11155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:51.446398 11155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:51.450032 11155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "0d2063ed98c54efcba1206c38f1151aa"
format_stamp: "Formatted at 2026-08-12 02:48:51 on dist-test-slave-ss4n"
server_key: "c5c748ba7265a541318204b7a5ff4258"
server_key_iv: "15c072e609de9c2b8ae454b0f067bc82"
server_key_version: "encryptionkey@0"
I20260812 02:48:51.450910 11155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "0d2063ed98c54efcba1206c38f1151aa"
format_stamp: "Formatted at 2026-08-12 02:48:51 on dist-test-slave-ss4n"
server_key: "c5c748ba7265a541318204b7a5ff4258"
server_key_iv: "15c072e609de9c2b8ae454b0f067bc82"
server_key_version: "encryptionkey@0"
I20260812 02:48:51.456916 11155 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:48:51.461055 11178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:51.462806 11155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260812 02:48:51.463073 11155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "0d2063ed98c54efcba1206c38f1151aa"
format_stamp: "Formatted at 2026-08-12 02:48:51 on dist-test-slave-ss4n"
server_key: "c5c748ba7265a541318204b7a5ff4258"
server_key_iv: "15c072e609de9c2b8ae454b0f067bc82"
server_key_version: "encryptionkey@0"
I20260812 02:48:51.463344 11155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:51.470363 11155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:51.475248 11155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:51.475653 11155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:51.496366 11155 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:45951
I20260812 02:48:51.496464 11236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:45951 every 8 connection(s)
I20260812 02:48:51.497968 11155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:51.499589 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 11155
I20260812 02:48:51.499785 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:51.500252 18195 external_mini_cluster.cc:1428] Setting key efed6290584f8f6b1ba82e9d8fd56872
I20260812 02:48:51.502769 11237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:51.509814 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:51.502229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36521 (local address 127.17.196.254:45951)
0812 02:48:51.503123 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:51.503138 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:51.503193 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:48:51.505649 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:51.505679 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:51.506158 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:51.506658 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:51.507717 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:51.507729 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:51.507752 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:51.507769 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:51.507858 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:51.507890 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:51.508135 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:51.508270 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 02:48:51.508473 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":348,"threads_started":1}
I20260812 02:48:51.514235 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:51.518524 11237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa: Bootstrap starting.
I20260812 02:48:51.523138 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:51.520072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49795 (local address 127.17.196.254:45951)
0812 02:48:51.520344 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:51.520355 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:51.520881 (+ 526us) server_negotiation.cc:407] Connection header received
0812 02:48:51.521086 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:51.521090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:51.521176 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:51.521308 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:51.522397 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:51.522403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:51.522408 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:51.522431 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:51.522479 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:51.522501 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:51.522639 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:51.522759 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 02:48:51.522828 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:48:51.523311 11237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:51.524946 11237 log.cc:824] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:51.528694 11237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa: No bootstrap required, opened a new log
I20260812 02:48:51.537719 11237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2063ed98c54efcba1206c38f1151aa" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45951 } }
I20260812 02:48:51.538166 11237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:51.538298 11237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2063ed98c54efcba1206c38f1151aa, State: Initialized, Role: FOLLOWER
I20260812 02:48:51.539036 11237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2063ed98c54efcba1206c38f1151aa" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45951 } }
I20260812 02:48:51.539381 11237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:51.539585 11237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:51.539750 11237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:51.542271 11237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2063ed98c54efcba1206c38f1151aa" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45951 } }
I20260812 02:48:51.542789 11237 leader_election.cc:302] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d2063ed98c54efcba1206c38f1151aa; no voters:
I20260812 02:48:51.543431 11237 leader_election.cc:288] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:51.543560 11249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:51.544435 11249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [term 1 LEADER]: Becoming Leader. State: Replica: 0d2063ed98c54efcba1206c38f1151aa, State: Running, Role: LEADER
I20260812 02:48:51.545079 11249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2063ed98c54efcba1206c38f1151aa" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45951 } }
I20260812 02:48:51.546083 11237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:51.548656 11250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d2063ed98c54efcba1206c38f1151aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2063ed98c54efcba1206c38f1151aa" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45951 } } }
I20260812 02:48:51.549036 11250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:51.548547 11251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d2063ed98c54efcba1206c38f1151aa. Latest consensus state: current_term: 1 leader_uuid: "0d2063ed98c54efcba1206c38f1151aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2063ed98c54efcba1206c38f1151aa" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45951 } } }
I20260812 02:48:51.549860 11251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:51.550000 11257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:51.555959 11257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:51.567133 11257 catalog_manager.cc:1347] Generated new cluster ID: 9c942477f1d84d3cb1b43d63bdfa61fd
I20260812 02:48:51.567291 11257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:51.580698 11257 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:51.581176 11257 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:51.592662 11257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d2063ed98c54efcba1206c38f1151aa: Generated new TSK 0
I20260812 02:48:51.593755 11257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:51.601630 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11155
W20260812 02:48:51.603066 11229 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36521
W20260812 02:48:51.604035 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11155: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:51.604249 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 11155
2026-08-12T02:48:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2264 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:51 dist-test-slave-ss4n krb5kdc[11272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:51 dist-test-slave-ss4n krb5kdc[11272](info): set up 2 sockets
Aug 12 02:48:51 dist-test-slave-ss4n krb5kdc[11272](info): commencing operation
krb5kdc: starting...
W20260812 02:48:52.947302 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:53 dist-test-slave-ss4n krb5kdc[11272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:48:53.326396 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:45767
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:40003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:45767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:53.696267 11286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:53.696722 11286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:53.696913 11286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:53.704583 11286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:53.704715 11286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:53.704813 11286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:53.704922 11286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:53.705003 11286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:53.715569 11286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:40003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:45767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:45767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:53.717123 11286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:53.718580 11286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:53.731945 11294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:53.731909 11291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:53.732111 11292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:53.732640 11286 server_base.cc:1034] running on GCE node
I20260812 02:48:53.733704 11286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:53.735263 11286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:53.736523 11286 hybrid_clock.cc:648] HybridClock initialized: now 1786502933736451 us; error 67 us; skew 500 ppm
I20260812 02:48:53.736939 11286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:53 dist-test-slave-ss4n krb5kdc[11272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502933, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:53.746130 11286 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:48:53.748049 11286 webserver.cc:460] Webserver started at http://127.17.196.254:42519/ using document root <none> and password file <none>
I20260812 02:48:53.748883 11286 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:53.749055 11286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:53.749404 11286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:53.752827 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "646f8524212a4141b00b14b7b7a5a742"
format_stamp: "Formatted at 2026-08-12 02:48:53 on dist-test-slave-ss4n"
server_key: "e2899e263a332fcee96f176642c97257"
server_key_iv: "61e9467bdcb08b6fdb1d79675aed2f62"
server_key_version: "encryptionkey@0"
I20260812 02:48:53.753754 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "646f8524212a4141b00b14b7b7a5a742"
format_stamp: "Formatted at 2026-08-12 02:48:53 on dist-test-slave-ss4n"
server_key: "e2899e263a332fcee96f176642c97257"
server_key_iv: "61e9467bdcb08b6fdb1d79675aed2f62"
server_key_version: "encryptionkey@0"
I20260812 02:48:53.760821 11286 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260812 02:48:53.765111 11302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:53.766697 11286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 02:48:53.766911 11286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "646f8524212a4141b00b14b7b7a5a742"
format_stamp: "Formatted at 2026-08-12 02:48:53 on dist-test-slave-ss4n"
server_key: "e2899e263a332fcee96f176642c97257"
server_key_iv: "61e9467bdcb08b6fdb1d79675aed2f62"
server_key_version: "encryptionkey@0"
I20260812 02:48:53.767175 11286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:53.807680 11286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:53.813156 11286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:53.813536 11286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:53.836417 11286 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:45767
I20260812 02:48:53.836464 11364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:45767 every 8 connection(s)
I20260812 02:48:53.838109 11286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:53.842367 11365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:53.844930 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 11286
I20260812 02:48:53.845140 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:53.845719 18195 external_mini_cluster.cc:1428] Setting key c8a3b40c101905e4c3453d4c68e3587d
I20260812 02:48:53.856645 11365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742: Bootstrap starting.
I20260812 02:48:53.860880 11365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:53.862347 11365 log.cc:824] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:48:53 dist-test-slave-ss4n krb5kdc[11272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:48:53.865947 11365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742: No bootstrap required, opened a new log
I20260812 02:48:53.873070 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:53.847889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35073 (local address 127.17.196.254:45767)
0812 02:48:53.849399 (+ 1510us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:53.849413 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:53.849467 (+ 54us) server_negotiation.cc:407] Connection header received
0812 02:48:53.851387 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:53.851414 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:53.851856 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:53.852197 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:53.854101 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:53.855630 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:53.857526 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:53.858292 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:53.858451 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:53.863783 (+ 5332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:53.863792 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:53.863809 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:53.863856 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:53.867701 (+ 3845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:53.868698 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:53.868712 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:53.868721 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:53.868932 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:53.869398 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:53.869404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:53.869412 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:53.870146 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:53.870573 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:53.871169 (+ 596us) server_negotiation.cc:299] Negotiation successful
0812 02:48:53.871876 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1281,"thread_start_us":1028,"threads_started":1}
I20260812 02:48:53.874784 11365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646f8524212a4141b00b14b7b7a5a742" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45767 } }
I20260812 02:48:53.875214 11365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:53.875344 11365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646f8524212a4141b00b14b7b7a5a742, State: Initialized, Role: FOLLOWER
I20260812 02:48:53.876073 11365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646f8524212a4141b00b14b7b7a5a742" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45767 } }
I20260812 02:48:53.876389 11365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:53.876566 11365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:53.876771 11365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:53.877913 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:53.879468 11365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646f8524212a4141b00b14b7b7a5a742" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45767 } }
I20260812 02:48:53.880046 11365 leader_election.cc:302] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 646f8524212a4141b00b14b7b7a5a742; no voters:
I20260812 02:48:53.880770 11365 leader_election.cc:288] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:53.880890 11373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:53.881933 11373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [term 1 LEADER]: Becoming Leader. State: Replica: 646f8524212a4141b00b14b7b7a5a742, State: Running, Role: LEADER
I20260812 02:48:53.882810 11373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646f8524212a4141b00b14b7b7a5a742" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45767 } }
I20260812 02:48:53.884491 11365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:53.886296 11379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "646f8524212a4141b00b14b7b7a5a742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646f8524212a4141b00b14b7b7a5a742" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45767 } } }
I20260812 02:48:53.886333 11376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 646f8524212a4141b00b14b7b7a5a742. Latest consensus state: current_term: 1 leader_uuid: "646f8524212a4141b00b14b7b7a5a742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646f8524212a4141b00b14b7b7a5a742" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 45767 } } }
I20260812 02:48:53.887085 11376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:53.887080 11379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742 [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:53.903676 11386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:53.904392 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:53.883549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34375 (local address 127.17.196.254:45767)
0812 02:48:53.883733 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:53.883739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:53.883761 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:48:53.884033 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:53.884038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:53.884131 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:53.884200 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:53.886672 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:53.888577 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:53.891142 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:53.892119 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:53.892251 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:53.896965 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:53.896976 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:53.896984 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:48:53.897059 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:53.900719 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:53.901530 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:53.901536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:53.901539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:53.901675 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:48:53.902204 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:48:53.902209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:48:53.902211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:48:53.902653 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:53.902980 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:53.903443 (+ 463us) server_negotiation.cc:299] Negotiation successful
0812 02:48:53.903816 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:48:53.910442 11386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:53.924670 11386 catalog_manager.cc:1347] Generated new cluster ID: 102136be27b34fe59b5640b6fa6f3c0a
I20260812 02:48:53.924806 11386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:53.943940 11386 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:53.945407 11386 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:53.953320 11386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 646f8524212a4141b00b14b7b7a5a742: Generated new TSK 0
I20260812 02:48:53.954355 11386 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:53.965530 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11286
I20260812 02:48:53.966727 11359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:48:54.063367 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 11286
2026-08-12T02:48:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:54 dist-test-slave-ss4n krb5kdc[11407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:54 dist-test-slave-ss4n krb5kdc[11407](info): set up 2 sockets
Aug 12 02:48:54 dist-test-slave-ss4n krb5kdc[11407](info): commencing operation
krb5kdc: starting...
W20260812 02:48:55.371943 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:55 dist-test-slave-ss4n krb5kdc[11407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:48:55.753374 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46593
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:46593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:56.140178 11423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:56.140501 11423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:56.140674 11423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:56.148380 11423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:56.148514 11423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:56.148607 11423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:56.148712 11423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:56.148823 11423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:56.159293 11423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:46593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:46593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:56.160733 11423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:56.162230 11423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:56.175252 11429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:56.175426 11430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:56.175271 11432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:56.179735 11423 server_base.cc:1034] running on GCE node
I20260812 02:48:56.180751 11423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:56.182690 11423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:56.183943 11423 hybrid_clock.cc:648] HybridClock initialized: now 1786502936183895 us; error 37 us; skew 500 ppm
I20260812 02:48:56.184371 11423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:56 dist-test-slave-ss4n krb5kdc[11407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502936, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:56.188067 11423 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:48:56.189949 11423 webserver.cc:460] Webserver started at http://127.17.196.254:40511/ using document root <none> and password file <none>
I20260812 02:48:56.190779 11423 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:56.190936 11423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:56.191304 11423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:56.194821 11423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "d9a91a18dd814796a387a2745e8809bc"
format_stamp: "Formatted at 2026-08-12 02:48:56 on dist-test-slave-ss4n"
server_key: "f1c07b14197154b13cffde4cb406684b"
server_key_iv: "fa2741aac6b2fa903ec3ff2c62b63949"
server_key_version: "encryptionkey@0"
I20260812 02:48:56.195770 11423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "d9a91a18dd814796a387a2745e8809bc"
format_stamp: "Formatted at 2026-08-12 02:48:56 on dist-test-slave-ss4n"
server_key: "f1c07b14197154b13cffde4cb406684b"
server_key_iv: "fa2741aac6b2fa903ec3ff2c62b63949"
server_key_version: "encryptionkey@0"
I20260812 02:48:56.202090 11423 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 02:48:56.206481 11439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:56.208105 11423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:48:56.208340 11423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "d9a91a18dd814796a387a2745e8809bc"
format_stamp: "Formatted at 2026-08-12 02:48:56 on dist-test-slave-ss4n"
server_key: "f1c07b14197154b13cffde4cb406684b"
server_key_iv: "fa2741aac6b2fa903ec3ff2c62b63949"
server_key_version: "encryptionkey@0"
I20260812 02:48:56.208642 11423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:56.244506 11423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:56.249842 11423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:56.250182 11423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:56.273098 11423 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:46593
I20260812 02:48:56.273165 11505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:46593 every 8 connection(s)
I20260812 02:48:56.274777 11423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:56.279240 11506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:56.281876 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 11423
I20260812 02:48:56.282120 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:56.282639 18195 external_mini_cluster.cc:1428] Setting key dbea513e335b7e9b16d5f4669e2c4261
I20260812 02:48:56.293684 11506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc: Bootstrap starting.
I20260812 02:48:56.297839 11506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:56.299122 11506 log.cc:824] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:56.299919 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:56.285132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48433 (local address 127.17.196.254:46593)
0812 02:48:56.286583 (+ 1451us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:56.286597 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:56.286646 (+ 49us) server_negotiation.cc:407] Connection header received
0812 02:48:56.288564 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:56.288594 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:56.289074 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:56.289467 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:56.290292 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:56.292525 (+ 2233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:56.294608 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:56.295428 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:56.295606 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:56.297064 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:56.297073 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:56.297090 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:56.297791 (+ 701us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:56.297888 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:56.297918 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:56.298083 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:56.298402 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 02:48:56.298707 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":836,"threads_started":1}
I20260812 02:48:56.302389 11506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc: No bootstrap required, opened a new log
I20260812 02:48:56.304247 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:56.313685 11506 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9a91a18dd814796a387a2745e8809bc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46593 } }
I20260812 02:48:56.314154 11506 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:56.314275 11506 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a91a18dd814796a387a2745e8809bc, State: Initialized, Role: FOLLOWER
I20260812 02:48:56.314981 11506 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9a91a18dd814796a387a2745e8809bc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46593 } }
I20260812 02:48:56.315285 11506 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:56.315445 11506 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:56.315658 11506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:56.317965 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:56.310216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45103 (local address 127.17.196.254:46593)
0812 02:48:56.310473 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:56.310481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:56.310987 (+ 506us) server_negotiation.cc:407] Connection header received
0812 02:48:56.311189 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:56.311196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:56.311291 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:56.311420 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:56.312002 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:56.313106 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:56.315008 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:48:56.315727 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:48:56.315879 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:48:56.317130 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:56.317138 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:56.317143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:56.317167 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:56.317218 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:56.317245 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:56.317349 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:56.317514 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 02:48:56.317625 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:48:56.319067 11506 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9a91a18dd814796a387a2745e8809bc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46593 } }
I20260812 02:48:56.319772 11506 leader_election.cc:302] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9a91a18dd814796a387a2745e8809bc; no voters:
I20260812 02:48:56.320470 11506 leader_election.cc:288] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:56.320708 11516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:56.322904 11506 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:56.327847 11516 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [term 1 LEADER]: Becoming Leader. State: Replica: d9a91a18dd814796a387a2745e8809bc, State: Running, Role: LEADER
I20260812 02:48:56.328663 11516 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9a91a18dd814796a387a2745e8809bc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46593 } }
I20260812 02:48:56.332381 11518 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9a91a18dd814796a387a2745e8809bc. Latest consensus state: current_term: 1 leader_uuid: "d9a91a18dd814796a387a2745e8809bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9a91a18dd814796a387a2745e8809bc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46593 } } }
I20260812 02:48:56.332407 11517 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9a91a18dd814796a387a2745e8809bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9a91a18dd814796a387a2745e8809bc" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 46593 } } }
I20260812 02:48:56.333050 11517 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:56.333010 11518 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc [sys.catalog]: This master's current role is: LEADER
W20260812 02:48:56.338702 11533 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:48:56.338863 11533 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:48:56.339006 11534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:56.340939 11534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:56.350942 11534 catalog_manager.cc:1347] Generated new cluster ID: b2f102436038419184ce5194b2cbda86
I20260812 02:48:56.351090 11534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:56.379141 11534 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:56.381138 11534 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:56.396260 11534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9a91a18dd814796a387a2745e8809bc: Generated new TSK 0
I20260812 02:48:56.397447 11534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:56.407497 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11423
W20260812 02:48:56.409973 11500 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48433
W20260812 02:48:56.410938 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11423: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:56.411116 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 11423
2026-08-12T02:48:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:56 dist-test-slave-ss4n krb5kdc[11543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:56 dist-test-slave-ss4n krb5kdc[11543](info): set up 2 sockets
Aug 12 02:48:56 dist-test-slave-ss4n krb5kdc[11543](info): commencing operation
krb5kdc: starting...
W20260812 02:48:57.707237 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:48:57 dist-test-slave-ss4n krb5kdc[11543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:48:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:48:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:48:58.090024 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42967
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:45905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:48:58.463236 11559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:48:58.463572 11559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:48:58.463676 11559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:48:58.471331 11559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:48:58.471477 11559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:48:58.471525 11559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:48:58.471580 11559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:48:58.471618 11559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:48:58.482530 11559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:45905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:42967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:48:58.483909 11559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:48:58.485255 11559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:48:58.498641 11570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:58.498772 11565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:48:58.498865 11566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:48:58.499437 11559 server_base.cc:1034] running on GCE node
I20260812 02:48:58.500314 11559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:48:58.501986 11559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:48:58.503248 11559 hybrid_clock.cc:648] HybridClock initialized: now 1786502938503211 us; error 41 us; skew 500 ppm
I20260812 02:48:58.503731 11559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:48:58 dist-test-slave-ss4n krb5kdc[11543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502938, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:48:58.507361 11559 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:48:58.509102 11559 webserver.cc:460] Webserver started at http://127.17.196.254:36017/ using document root <none> and password file <none>
I20260812 02:48:58.509933 11559 fs_manager.cc:359] Metadata directory not provided
I20260812 02:48:58.510100 11559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:48:58.510421 11559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:48:58.513749 11559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd"
format_stamp: "Formatted at 2026-08-12 02:48:58 on dist-test-slave-ss4n"
server_key: "8039603481211789da1632ab070a1aa9"
server_key_iv: "386947fab7953ce11868d80ebfd14d73"
server_key_version: "encryptionkey@0"
I20260812 02:48:58.514608 11559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd"
format_stamp: "Formatted at 2026-08-12 02:48:58 on dist-test-slave-ss4n"
server_key: "8039603481211789da1632ab070a1aa9"
server_key_iv: "386947fab7953ce11868d80ebfd14d73"
server_key_version: "encryptionkey@0"
I20260812 02:48:58.520323 11559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:48:58.524442 11577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:48:58.526192 11559 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 02:48:58.526407 11559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd"
format_stamp: "Formatted at 2026-08-12 02:48:58 on dist-test-slave-ss4n"
server_key: "8039603481211789da1632ab070a1aa9"
server_key_iv: "386947fab7953ce11868d80ebfd14d73"
server_key_version: "encryptionkey@0"
I20260812 02:48:58.526669 11559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:48:58.533002 11559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:48:58.537447 11559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:48:58.537942 11559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:48:58.558801 11559 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:42967
I20260812 02:48:58.558835 11635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:42967 every 8 connection(s)
I20260812 02:48:58.560267 11559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:48:58.564714 11636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:48:58.567574 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 11559
I20260812 02:48:58.567795 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:48:58.568253 18195 external_mini_cluster.cc:1428] Setting key aa134a1eab0b3da3f03c18812d203083
I20260812 02:48:58.576831 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:58.570382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46889 (local address 127.17.196.254:42967)
0812 02:48:58.571370 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:58.571383 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:58.571428 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:48:58.573148 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:58.573170 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:58.573604 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:58.573990 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:58.575124 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:58.575131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:58.575144 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:58.575163 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:58.575231 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:58.575258 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:58.575506 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:58.575710 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 02:48:58.575888 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":512,"threads_started":1}
I20260812 02:48:58.579031 11636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd: Bootstrap starting.
I20260812 02:48:58.580789 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:48:58.583421 11636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd: Neither blocks nor log segments found. Creating new log.
I20260812 02:48:58.585234 11636 log.cc:824] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd: Log is configured to *not* fsync() on all Append() calls
I20260812 02:48:58.589273 11636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd: No bootstrap required, opened a new log
I20260812 02:48:58.590379 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:48:58.587108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38747 (local address 127.17.196.254:42967)
0812 02:48:58.587356 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 02:48:58.587369 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:48:58.587925 (+ 556us) server_negotiation.cc:407] Connection header received
0812 02:48:58.588141 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:48:58.588146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:48:58.588197 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:48:58.588286 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:48:58.589362 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:48:58.589367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:48:58.589371 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:48:58.589435 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:48:58.589532 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:48:58.589562 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:48:58.589695 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 02:48:58.589899 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 02:48:58.590030 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:48:58.598140 11636 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42967 } }
I20260812 02:48:58.598676 11636 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:48:58.598814 11636 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce8c1d9c3fef4acb85d93e384b64d9cd, State: Initialized, Role: FOLLOWER
I20260812 02:48:58.599699 11636 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42967 } }
I20260812 02:48:58.599937 11636 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:48:58.600044 11636 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:48:58.600178 11636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:48:58.602680 11636 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42967 } }
I20260812 02:48:58.603181 11636 leader_election.cc:302] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce8c1d9c3fef4acb85d93e384b64d9cd; no voters:
I20260812 02:48:58.603826 11636 leader_election.cc:288] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:48:58.603974 11648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:48:58.605568 11648 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [term 1 LEADER]: Becoming Leader. State: Replica: ce8c1d9c3fef4acb85d93e384b64d9cd, State: Running, Role: LEADER
I20260812 02:48:58.606166 11648 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42967 } }
I20260812 02:48:58.606427 11636 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:48:58.609440 11649 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42967 } } }
I20260812 02:48:58.609999 11649 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:58.609467 11650 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce8c1d9c3fef4acb85d93e384b64d9cd. Latest consensus state: current_term: 1 leader_uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8c1d9c3fef4acb85d93e384b64d9cd" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 42967 } } }
I20260812 02:48:58.610623 11650 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd [sys.catalog]: This master's current role is: LEADER
I20260812 02:48:58.614826 11660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:48:58.620702 11660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:48:58.632061 11660 catalog_manager.cc:1347] Generated new cluster ID: d533fa762cd7406e9dbea962d8c916ee
I20260812 02:48:58.632197 11660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:48:58.656549 11660 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:48:58.657101 11660 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:48:58.668818 11660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce8c1d9c3fef4acb85d93e384b64d9cd: Generated new TSK 0
I20260812 02:48:58.669905 11660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:48:58.689471 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11559
W20260812 02:48:58.691135 11631 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46889
W20260812 02:48:58.692054 18195 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11559: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:48:58.692240 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 11559
2026-08-12T02:48:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2278 ms)
[----------] 18 tests from AuthTokenIssuingTest (42956 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:48:58 dist-test-slave-ss4n krb5kdc[11675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:48:58 dist-test-slave-ss4n krb5kdc[11675](info): set up 2 sockets
Aug 12 02:48:58 dist-test-slave-ss4n krb5kdc[11675](info): commencing operation
krb5kdc: starting...
W20260812 02:48:59.983214 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.003s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:49:00 dist-test-slave-ss4n krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:49:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:49:00.374416 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:34373
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:34373,127.17.196.253:41733,127.17.196.252:40169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:49:00.748242 11691 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:49:00.748646 11691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:49:00.748729 11691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:49:00.748831 11691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:49:00.757092 11691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:49:00.757215 11691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:49:00.757273 11691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:49:00.757319 11691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:49:00.757385 11691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:49:00.767831 11691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:34373,127.17.196.253:41733,127.17.196.252:40169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:34373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:49:00.769236 11691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:49:00.770709 11691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:49:00.783878 11697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:00.783859 11696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:00.783982 11699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:49:00.785506 11691 server_base.cc:1034] running on GCE node
I20260812 02:49:00.786620 11691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:49:00.788488 11691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:49:00.789817 11691 hybrid_clock.cc:648] HybridClock initialized: now 1786502940789747 us; error 53 us; skew 500 ppm
I20260812 02:49:00.790212 11691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:49:00 dist-test-slave-ss4n krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:49:00.794121 11691 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:49:00.795965 11691 webserver.cc:460] Webserver started at http://127.17.196.254:44135/ using document root <none> and password file <none>
I20260812 02:49:00.796747 11691 fs_manager.cc:359] Metadata directory not provided
I20260812 02:49:00.796846 11691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:49:00.797200 11691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:49:00.800772 11691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6"
format_stamp: "Formatted at 2026-08-12 02:49:00 on dist-test-slave-ss4n"
server_key: "85546a87225100316cdc6b9a92b7c1d2"
server_key_iv: "b2a3f5b366a8ead31375762ae4dfebf8"
server_key_version: "encryptionkey@0"
I20260812 02:49:00.801810 11691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6"
format_stamp: "Formatted at 2026-08-12 02:49:00 on dist-test-slave-ss4n"
server_key: "85546a87225100316cdc6b9a92b7c1d2"
server_key_iv: "b2a3f5b366a8ead31375762ae4dfebf8"
server_key_version: "encryptionkey@0"
I20260812 02:49:00.807602 11691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:49:00.811406 11706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:49:00.813078 11691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:49:00.813344 11691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6"
format_stamp: "Formatted at 2026-08-12 02:49:00 on dist-test-slave-ss4n"
server_key: "85546a87225100316cdc6b9a92b7c1d2"
server_key_iv: "b2a3f5b366a8ead31375762ae4dfebf8"
server_key_version: "encryptionkey@0"
I20260812 02:49:00.813753 11691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:49:00.844304 11691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:49:00.849234 11691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:49:00.849566 11691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:49:00.870995 11691 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:34373
I20260812 02:49:00.871014 11769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:34373 every 8 connection(s)
I20260812 02:49:00.872581 11691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:49:00.876857 11770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:49:00.882226 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 11691
I20260812 02:49:00.882480 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:49:00.883149 18195 external_mini_cluster.cc:1428] Setting key af7e40ad087b2a1b46f641b0b89debf8
I20260812 02:49:00.884562 11770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:00 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:49:00.912840 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:00.888355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34373 (local address 127.0.0.1:34058)
0812 02:49:00.889838 (+ 1483us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:00.889887 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:00.892208 (+ 2321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:00.893334 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:00.893340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:00.894136 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:00.894542 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:00.894563 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:00.896965 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:00.896970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:00.898379 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:00.898395 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:00.899753 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:00.899764 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:00.900047 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:00.900926 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:00.900974 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:00.904310 (+ 3336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:00.908062 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:00.908078 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:00.908092 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:00.908548 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:00.909020 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:00.909027 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:00.909031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:00.909206 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:00.910119 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:00.910129 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:00.910389 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 02:49:00.910676 (+ 287us) client_negotiation.cc:240] Negotiation successful
0812 02:49:00.911404 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"mutex_wait_us":198,"spinlock_wait_cycles":72448,"thread_start_us":914,"threads_started":1}
I20260812 02:49:00.912859 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:00.888355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.17.196.254:34373)
0812 02:49:00.889353 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:00.889371 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:00.890062 (+ 691us) server_negotiation.cc:407] Connection header received
0812 02:49:00.892685 (+ 2623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:00.892697 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:00.893041 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:00.893280 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:00.894792 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:00.896753 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:00.898560 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:00.899451 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:00.899604 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:00.904614 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:00.904624 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:00.904640 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:00.904676 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:00.907877 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:00.908724 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:00.908731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:00.908734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:00.908891 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:00.909330 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:00.909335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:00.909342 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:00.909973 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:00.910218 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:00.910822 (+ 604us) server_negotiation.cc:299] Negotiation successful
0812 02:49:00.911267 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":709,"thread_start_us":429,"threads_started":1}
I20260812 02:49:00.916290 11770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:49:00.917780 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:00.917108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:39938)
0812 02:49:00.917365 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:00.917576 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260812 02:49:00.918068 11712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.196.253@KRBTEST.COM; defaulting to no policy
I20260812 02:49:00.919967 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:00.919471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:39940)
0812 02:49:00.919684 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:00.919768 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260812 02:49:00.920248 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:41733: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab.
Entry for principal kudu/127.17.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab.
I20260812 02:49:00.966830 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.253:41733
--webserver_interface=127.17.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab
--principal=kudu/127.17.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:34373,127.17.196.253:41733,127.17.196.252:40169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:49:00.969682 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } attempt: 1
I20260812 02:49:00.971045 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:00.970500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:39954)
0812 02:49:00.970727 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:00.970880 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 02:49:00.972252 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:00.971809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:39962)
0812 02:49:00.972021 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:00.972100 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 02:49:00.972465 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:41733: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
I20260812 02:49:01.040781 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } attempt: 2
I20260812 02:49:01.042399 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.041732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:39974)
0812 02:49:01.042047 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.042203 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 02:49:01.043717 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.043204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:39982)
0812 02:49:01.043415 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.043533 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 02:49:01.043982 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:41733: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
I20260812 02:49:01.135278 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } attempt: 3
I20260812 02:49:01.136647 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.136049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:39992)
0812 02:49:01.136336 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.136469 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 02:49:01.138044 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.137428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:40000)
0812 02:49:01.137675 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.137796 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260812 02:49:01.138298 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:41733: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
I20260812 02:49:01.281731 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } attempt: 4
I20260812 02:49:01.283134 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.282471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:40012)
0812 02:49:01.282804 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.282935 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260812 02:49:01.284442 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.283960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:40020)
0812 02:49:01.284166 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.284274 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 02:49:01.284754 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:41733: Network error: Client connection negotiation failed: client connection to 127.17.196.253:41733: connect: Connection refused (error 111)
W20260812 02:49:01.344152 11778 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:49:01.344606 11778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:49:01.344750 11778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:49:01.344898 11778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:49:01.352778 11778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:49:01.352929 11778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:49:01.353042 11778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:49:01.353142 11778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:49:01.353266 11778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:49:01.363795 11778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:34373,127.17.196.253:41733,127.17.196.252:40169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab
--principal=kudu/127.17.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.253:41733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:49:01.365263 11778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:49:01.366699 11778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:49:01.379844 11785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:01.380030 11787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:01.380455 11784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:49:01.380918 11778 server_base.cc:1034] running on GCE node
I20260812 02:49:01.381932 11778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:49:01.383474 11778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:49:01.384737 11778 hybrid_clock.cc:648] HybridClock initialized: now 1786502941384662 us; error 41 us; skew 500 ppm
I20260812 02:49:01.385146 11778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:49:01 dist-test-slave-ss4n krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502941, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:49:01.388831 11778 init.cc:377] Logged in from keytab as kudu/127.17.196.253@KRBTEST.COM (short username kudu)
I20260812 02:49:01.390506 11778 webserver.cc:460] Webserver started at http://127.17.196.253:39773/ using document root <none> and password file <none>
I20260812 02:49:01.391295 11778 fs_manager.cc:359] Metadata directory not provided
I20260812 02:49:01.391441 11778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:49:01.391798 11778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:49:01.395105 11778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/data/instance:
uuid: "163c0e6dece14590bf2fa25faa4ac869"
format_stamp: "Formatted at 2026-08-12 02:49:01 on dist-test-slave-ss4n"
server_key: "a268094419143811d066ba9e348a8fad"
server_key_iv: "e9fe9f279a72da2cd844455585c99543"
server_key_version: "encryptionkey@0"
I20260812 02:49:01.396045 11778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/wal/instance:
uuid: "163c0e6dece14590bf2fa25faa4ac869"
format_stamp: "Formatted at 2026-08-12 02:49:01 on dist-test-slave-ss4n"
server_key: "a268094419143811d066ba9e348a8fad"
server_key_iv: "e9fe9f279a72da2cd844455585c99543"
server_key_version: "encryptionkey@0"
I20260812 02:49:01.401855 11778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 02:49:01.405879 11795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:49:01.407392 11778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:49:01.407613 11778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/wal
uuid: "163c0e6dece14590bf2fa25faa4ac869"
format_stamp: "Formatted at 2026-08-12 02:49:01 on dist-test-slave-ss4n"
server_key: "a268094419143811d066ba9e348a8fad"
server_key_iv: "e9fe9f279a72da2cd844455585c99543"
server_key_version: "encryptionkey@0"
I20260812 02:49:01.407866 11778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:49:01.427434 11778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:49:01.432754 11778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:49:01.433081 11778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:49:01.454221 11778 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.253:41733
I20260812 02:49:01.454245 11854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.253:41733 every 8 connection(s)
I20260812 02:49:01.455734 11778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
I20260812 02:49:01.460081 11855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:49:01.464579 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 11778
I20260812 02:49:01.464839 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-1/wal/instance
I20260812 02:49:01.465293 18195 external_mini_cluster.cc:1428] Setting key 8842236e333e123bfa4c90b41ea0a587
I20260812 02:49:01.468179 11855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:01 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502941, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:49:01.491415 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.471546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.17.196.254:34373)
0812 02:49:01.472134 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:01.472139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:01.473294 (+ 1155us) server_negotiation.cc:407] Connection header received
0812 02:49:01.475474 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:01.475480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:01.475589 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:01.475727 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:01.477529 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.478335 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.479991 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.480432 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.480598 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:01.484770 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:01.484776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:01.484779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:01.484822 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:01.487782 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:01.488762 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:01.488768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:01.488771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:01.488910 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:01.489383 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:01.489389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:01.489392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:01.489775 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:01.489960 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:01.490753 (+ 793us) server_negotiation.cc:299] Negotiation successful
0812 02:49:01.491005 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":338,"threads_started":1}
I20260812 02:49:01.492558 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.472210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34373 (local address 127.0.0.1:34072)
0812 02:49:01.473080 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.473132 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:01.475144 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:01.476012 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:01.476020 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:01.476595 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:01.477237 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:01.477256 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.478535 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.478539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:01.479795 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:01.479819 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.480696 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.480704 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:01.480968 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:01.481846 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:01.481870 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:01.484487 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:01.488060 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:01.488070 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:01.488091 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:01.488614 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:01.489043 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:01.489048 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:01.489052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:01.489250 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:01.489931 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:01.489942 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:01.490357 (+ 415us) client_negotiation.cc:769] Sending connection context
0812 02:49:01.490666 (+ 309us) client_negotiation.cc:240] Negotiation successful
0812 02:49:01.491239 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":375,"threads_started":1}
I20260812 02:49:01.494576 11855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.252@KRBTEST.COM" created.
Aug 12 02:49:01 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502941, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:49:01.510582 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.495298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:40028)
0812 02:49:01.495448 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.495470 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:01.495598 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:01.496685 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:01.496693 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:01.497246 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:01.497516 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:01.497536 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.499270 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.499274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:01.500156 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:01.500168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.501203 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.501208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:01.501384 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:01.502164 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:01.502183 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:01.504298 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:01.507911 (+ 3613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:01.507917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:01.507921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:01.508310 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:01.508719 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:01.508725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:01.508728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:01.508873 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:01.509515 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:01.509520 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:01.509681 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 02:49:01.509829 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 02:49:01.510157 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 02:49:01.510718 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.495569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.17.196.253:41733)
0812 02:49:01.496202 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:01.496208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:01.496239 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:49:01.496285 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:01.496291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:01.496471 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:01.496604 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:01.497776 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.499106 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.500336 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.501016 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.501186 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:01.504533 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:01.504547 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:01.504564 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:01.504597 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:01.507728 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:01.508471 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:01.508479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:01.508481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:01.508603 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:01.508983 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:01.508989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:01.508992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:01.509360 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:01.509621 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:01.509989 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 02:49:01.510305 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":473,"thread_start_us":314,"threads_started":1}
I20260812 02:49:01.513710 11855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:49:01.515116 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.514513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51412)
0812 02:49:01.514808 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.514941 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260812 02:49:01.515481 11799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:01.517468 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.516975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51428)
0812 02:49:01.517177 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.517296 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 02:49:01.517782 11855 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab.
Entry for principal kudu/127.17.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab.
I20260812 02:49:01.546981 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.252:40169
--webserver_interface=127.17.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab
--principal=kudu/127.17.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:34373,127.17.196.253:41733,127.17.196.252:40169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:49:01.567250 11855 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 1
I20260812 02:49:01.568893 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.568291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51432)
0812 02:49:01.568576 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.568684 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260812 02:49:01.570490 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.569918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51448)
0812 02:49:01.570260 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.570354 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
W20260812 02:49:01.570851 11855 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:01.584055 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } attempt: 5
Aug 12 02:49:01 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:49:01.599648 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.585007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:40042)
0812 02:49:01.585366 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.585390 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:01.585669 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:01.586294 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:01.586300 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:01.586764 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:01.586949 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:01.586959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.588072 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.588079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:01.589045 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:01.589056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.590002 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.590009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:01.590262 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:01.591016 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:01.591036 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:01.593291 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:01.596884 (+ 3593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:01.596890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:01.596894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:01.597375 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:01.597826 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:01.597841 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:01.597844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:01.597994 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:01.598598 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:01.598603 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:01.598762 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 02:49:01.598921 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 02:49:01.599268 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260812 02:49:01.599646 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.585689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40042 (local address 127.17.196.253:41733)
0812 02:49:01.585906 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:01.585911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:01.585934 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:49:01.586027 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:01.586031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:01.586113 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:01.586247 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:01.587119 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.587907 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.589217 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:01.589796 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:01.589967 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:01.593562 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:01.593569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:01.593572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:01.593612 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:01.596690 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:01.597544 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:01.597550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:01.597554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:01.597695 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:01.598123 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:01.598128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:01.598131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:01.598451 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:01.598614 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:01.599027 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 02:49:01.599276 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:49:01.600816 11770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:49:01.602311 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.601853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51454)
0812 02:49:01.602038 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.602173 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 02:49:01.603394 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.603040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51458)
0812 02:49:01.603216 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.603282 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 02:49:01.603734 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:01.639142 11855 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 2
I20260812 02:49:01.640573 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.640027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51472)
0812 02:49:01.640281 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.640380 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 02:49:01.642063 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.641593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51478)
0812 02:49:01.641828 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.641919 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260812 02:49:01.642386 11855 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:01.655030 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 1
I20260812 02:49:01.656468 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.655919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51494)
0812 02:49:01.656179 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.656280 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 02:49:01.657756 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.657274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51498)
0812 02:49:01.657448 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.657627 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 02:49:01.658010 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:01.726258 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 2
I20260812 02:49:01.727721 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.727152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51504)
0812 02:49:01.727396 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.727501 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 02:49:01.728804 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.728426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51508)
0812 02:49:01.728598 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.728677 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 02:49:01.729029 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:01.733656 11855 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 3
I20260812 02:49:01.734985 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.734448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51512)
0812 02:49:01.734712 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.734824 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 02:49:01.736269 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.735763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51516)
0812 02:49:01.736052 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.736140 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260812 02:49:01.736495 11855 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:01.835253 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 3
I20260812 02:49:01.836541 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.836010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51520)
0812 02:49:01.836250 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.836350 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 02:49:01.837682 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.837231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:51536)
0812 02:49:01.837424 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.837568 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260812 02:49:01.837882 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:01.879736 11855 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 4
I20260812 02:49:01.881083 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.880493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:44502)
0812 02:49:01.880771 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.880887 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 02:49:01.882261 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:01.881883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:44512)
0812 02:49:01.882057 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:01.882135 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 02:49:01.882583 11855 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
W20260812 02:49:01.957620 11864 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:49:01.958029 11864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:49:01.958153 11864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:49:01.958290 11864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:49:01.965864 11864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:49:01.965960 11864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:49:01.966055 11864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:49:01.966152 11864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:49:01.966274 11864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:49:01.977583 11864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:34373,127.17.196.253:41733,127.17.196.252:40169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab
--principal=kudu/127.17.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.252:40169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:49:01.979043 11864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:49:01.980466 11864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:49:01.993901 11869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:01.994065 11873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:01.993954 11870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:49:01.995200 11864 server_base.cc:1034] running on GCE node
I20260812 02:49:01.996111 11864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:49:01.997828 11864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:49:01.999110 11864 hybrid_clock.cc:648] HybridClock initialized: now 1786502941999018 us; error 70 us; skew 500 ppm
I20260812 02:49:01.999579 11864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502942, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:49:02.003399 11864 init.cc:377] Logged in from keytab as kudu/127.17.196.252@KRBTEST.COM (short username kudu)
I20260812 02:49:02.005118 11864 webserver.cc:460] Webserver started at http://127.17.196.252:36677/ using document root <none> and password file <none>
I20260812 02:49:02.006016 11864 fs_manager.cc:359] Metadata directory not provided
I20260812 02:49:02.006175 11864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:49:02.006560 11864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:49:02.010145 11864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/data/instance:
uuid: "8b2f0cdda5c2418082ff3b9f20968c12"
format_stamp: "Formatted at 2026-08-12 02:49:02 on dist-test-slave-ss4n"
server_key: "ae21ffbf1f928b2d28227cac7f26fc87"
server_key_iv: "067d9ad4596981db13f1049b64063ef2"
server_key_version: "encryptionkey@0"
I20260812 02:49:02.011148 11864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/wal/instance:
uuid: "8b2f0cdda5c2418082ff3b9f20968c12"
format_stamp: "Formatted at 2026-08-12 02:49:02 on dist-test-slave-ss4n"
server_key: "ae21ffbf1f928b2d28227cac7f26fc87"
server_key_iv: "067d9ad4596981db13f1049b64063ef2"
server_key_version: "encryptionkey@0"
I20260812 02:49:02.015110 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 4
I20260812 02:49:02.016386 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.015814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:44518)
0812 02:49:02.016034 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:02.016171 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 02:49:02.017886 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.017370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:44520)
0812 02:49:02.017651 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:02.017740 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260812 02:49:02.018146 11770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:40169: Network error: Client connection negotiation failed: client connection to 127.17.196.252:40169: connect: Connection refused (error 111)
I20260812 02:49:02.018234 11864 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260812 02:49:02.022706 11883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:49:02.024210 11864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:49:02.024420 11864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/wal
uuid: "8b2f0cdda5c2418082ff3b9f20968c12"
format_stamp: "Formatted at 2026-08-12 02:49:02 on dist-test-slave-ss4n"
server_key: "ae21ffbf1f928b2d28227cac7f26fc87"
server_key_iv: "067d9ad4596981db13f1049b64063ef2"
server_key_version: "encryptionkey@0"
I20260812 02:49:02.024696 11864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:49:02.041976 11864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:49:02.047513 11864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:49:02.047852 11864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:49:02.068179 11864 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.252:40169
I20260812 02:49:02.068284 11940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.252:40169 every 8 connection(s)
I20260812 02:49:02.069774 11864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
I20260812 02:49:02.073807 11941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:49:02.078794 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 11864
I20260812 02:49:02.079000 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502547576936-18195-0/minicluster-data/master-2/wal/instance
I20260812 02:49:02.079564 18195 external_mini_cluster.cc:1428] Setting key 840bd59535b8a10702085686550cd6ad
I20260812 02:49:02.082000 11941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502942, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
I20260812 02:49:02.103900 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.084904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.17.196.254:34373)
0812 02:49:02.085555 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.085562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.086171 (+ 609us) server_negotiation.cc:407] Connection header received
0812 02:49:02.086351 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.086357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.086443 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.086573 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.088173 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.089002 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.090837 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.091278 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.091461 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.095421 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.095426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.095430 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.095467 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.099535 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.100660 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.100666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.100669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.100843 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.101472 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.101477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.101520 (+ 43us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.101934 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.102092 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.103219 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.103482 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":261,"threads_started":1}
I20260812 02:49:02.105264 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.084897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:34373 (local address 127.0.0.1:60220)
0812 02:49:02.085979 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:02.086027 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:02.086185 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:02.086664 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:02.086669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:02.087342 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:02.087937 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.087965 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.089193 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.089202 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.090609 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.090624 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.091584 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.091593 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.091926 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.092646 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:02.092670 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:02.095163 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:02.099713 (+ 4550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.099724 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.099734 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.100450 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.101048 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.101054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.101058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.101329 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.102096 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:02.102103 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:02.102340 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 02:49:02.102943 (+ 603us) client_negotiation.cc:240] Negotiation successful
0812 02:49:02.103578 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":584,"threads_started":1}
I20260812 02:49:02.105230 11944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.081826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.17.196.252:40169)
0812 02:49:02.082815 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.082827 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.082879 (+ 52us) server_negotiation.cc:407] Connection header received
0812 02:49:02.084972 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.085004 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.085644 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.086094 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.087712 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.089135 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.090609 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.091234 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.091451 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.095779 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.095792 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.095811 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.095848 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.099743 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.100667 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.100676 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.100679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.100874 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.101471 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.101476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.101480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.102315 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.102596 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.103219 (+ 623us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.103713 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":217,"server-negotiator.queue_time_us":760,"spinlock_wait_cycles":16256,"thread_start_us":512,"threads_started":1}
I20260812 02:49:02.107419 11941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502942, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:49:02.124881 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.108268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:41733 (local address 127.0.0.1:38526)
0812 02:49:02.108568 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:02.108591 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:02.108712 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:02.109741 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:02.109745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:02.110325 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:02.110594 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.110608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.112085 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.112089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.113688 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.113715 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.114561 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.114566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.114720 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.115399 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:02.115418 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:02.117770 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:02.121741 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.121748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.121752 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.122490 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.122971 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.122977 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.122981 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.123198 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.123793 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:02.123798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:02.123989 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 02:49:02.124149 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 02:49:02.124485 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260812 02:49:02.125002 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.108720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.17.196.253:41733)
0812 02:49:02.109282 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.109289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.109310 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:49:02.109376 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.109380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.109489 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.109698 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.110809 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.111901 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.113869 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.114381 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.114527 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.118068 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.118078 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.118082 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.118131 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.121557 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.122660 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.122665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.122667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.122812 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.123350 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.123354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.123356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.123647 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.123849 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.124243 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.124542 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":311,"threads_started":1}
I20260812 02:49:02.126387 11941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:49:02.127194 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.109083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.17.196.254:34373)
0812 02:49:02.109282 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.109289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.109435 (+ 146us) server_negotiation.cc:407] Connection header received
0812 02:49:02.109665 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.109672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.109759 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.109859 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.111498 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.112466 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.114354 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.114889 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.115043 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.119910 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.119916 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.119919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.119958 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.123444 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.124226 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.124233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.124237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.124387 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.124871 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.124877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.124881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.125610 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.125906 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.126483 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.126774 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502942, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:49:02.141681 11944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.127324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.17.196.252:40169)
0812 02:49:02.127887 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.127892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.128073 (+ 181us) server_negotiation.cc:407] Connection header received
0812 02:49:02.128276 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.128281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.128369 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.128498 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.129342 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.130245 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.131345 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.131754 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.131894 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.135237 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.135242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.135245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.135272 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.138679 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.139425 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.139433 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.139447 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.139592 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.140022 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.140029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.140033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.140348 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.140533 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.140936 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.141195 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":455}
I20260812 02:49:02.141773 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.127381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:44546)
0812 02:49:02.127973 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:02.127988 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:02.128140 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:02.128545 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:02.128550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:02.129001 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:02.129192 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.129205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.130368 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.130371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.131203 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.131213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.131904 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.131909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.132188 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.132826 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:02.132843 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:02.135046 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:02.138869 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.138875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.138880 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.139283 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.139722 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.139729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.139733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.139883 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.140505 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:02.140511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:02.140681 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 02:49:02.140852 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 02:49:02.141362 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":179}
I20260812 02:49:02.145260 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.129339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.17.196.253:41733)
0812 02:49:02.129526 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.129533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.129670 (+ 137us) server_negotiation.cc:407] Connection header received
0812 02:49:02.129805 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.129812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.129888 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.130011 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.131269 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.131976 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.133400 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.134052 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.134190 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.139251 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.139256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.139260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.139296 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.142392 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.143179 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.143185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.143188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.143333 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.143752 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.143756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.143759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.144060 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.144343 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.144651 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.144899 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:49:02.147746 11941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: Bootstrap starting.
I20260812 02:49:02.151401 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:49:02.152515 11941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: Neither blocks nor log segments found. Creating new log.
I20260812 02:49:02.154066 11941 log.cc:824] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: Log is configured to *not* fsync() on all Append() calls
I20260812 02:49:02.157924 11941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: No bootstrap required, opened a new log
I20260812 02:49:02.155539 11738 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60236
I20260812 02:49:02.170529 11941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } }
I20260812 02:49:02.171059 11941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2f0cdda5c2418082ff3b9f20968c12, State: Initialized, Role: FOLLOWER
I20260812 02:49:02.171684 11941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } }
I20260812 02:49:02.172796 11949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.173193 11949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:02.174340 11941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:49:02.181833 11855 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 5
W20260812 02:49:02.185259 11962 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:49:02.185557 11962 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502941, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
I20260812 02:49:02.182874 11864 master_runner.cc:186] Error getting master registration for 127.17.196.254:34373: OK, instance_id { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" instance_seqno: 1786502940864486 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:49:02.196453 11864 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:49:02.201326 11944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.183290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.17.196.252:40169)
0812 02:49:02.183467 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.183473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.183490 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:49:02.183547 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.183553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.183631 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.183745 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.184901 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.186167 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.187812 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.188681 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.188813 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.193372 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.193379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.193384 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.193437 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.197833 (+ 4396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.198844 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.198849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.198853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.199014 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.199548 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.199553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.199560 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.200015 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.200175 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.200688 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.200934 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:49:02.201418 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.182480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:44562)
0812 02:49:02.182908 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:02.182944 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:02.183066 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:02.183955 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:02.183963 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:02.184501 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:02.184718 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.184729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.186347 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.186353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.187635 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.187649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.189078 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.189084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.189347 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.190084 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:02.190108 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:02.193078 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:02.198028 (+ 4950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.198035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.198039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.198689 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.199155 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.199161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.199165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.199404 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.200167 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:02.200173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:02.200382 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 02:49:02.200575 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 02:49:02.200970 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260812 02:49:02.207916 11855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: Bootstrap starting.
I20260812 02:49:02.211511 11855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: Neither blocks nor log segments found. Creating new log.
I20260812 02:49:02.212843 11855 log.cc:824] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: Log is configured to *not* fsync() on all Append() calls
I20260812 02:49:02.215735 11855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: No bootstrap required, opened a new log
I20260812 02:49:02.224502 11855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } }
I20260812 02:49:02.224901 11855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163c0e6dece14590bf2fa25faa4ac869, State: Initialized, Role: FOLLOWER
I20260812 02:49:02.225411 11855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } }
I20260812 02:49:02.226531 11965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.227002 11965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:02.227788 11855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 02:49:02.239630 11978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:49:02.239880 11978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:49:02.235981 11778 master_runner.cc:186] Error getting master registration for 127.17.196.254:34373: OK, instance_id { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" instance_seqno: 1786502940864486 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:49:02.248778 11778 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:49:02.295423 11770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } attempt: 5
W20260812 02:49:02.300475 11890 tablet.cc:2369] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 02:49:02 dist-test-slave-ss4n krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
I20260812 02:49:02.316321 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.296284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:40169 (local address 127.0.0.1:44564)
0812 02:49:02.296545 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:02.296571 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:02.296712 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:02.297754 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:02.297761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:02.298377 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:02.298629 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.298644 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.300383 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.300389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.302136 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:02.302189 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.303609 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.303616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:02.303849 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.304660 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:02.304688 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:02.307905 (+ 3217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:02.313095 (+ 5190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.313103 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.313108 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.313756 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.314304 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:02.314309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:02.314313 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:02.314510 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:02.315200 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:02.315206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:02.315405 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 02:49:02.315574 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 02:49:02.315885 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 02:49:02.316493 11944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.296917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.17.196.252:40169)
0812 02:49:02.297135 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.297142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.297169 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:49:02.297253 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.297259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.297378 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.297531 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.298855 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.300183 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.302380 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.303206 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.303357 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.308179 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.308189 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.308195 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.308250 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.312872 (+ 4622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.313937 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.313952 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.313957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.314161 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.314661 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.314667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.314671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.315078 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.315248 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.315735 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.316038 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 02:49:02.323983 11770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6: Bootstrap starting.
I20260812 02:49:02.329085 11770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6: Neither blocks nor log segments found. Creating new log.
I20260812 02:49:02.331041 11770 log.cc:824] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6: Log is configured to *not* fsync() on all Append() calls
I20260812 02:49:02.335429 11770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6: No bootstrap required, opened a new log
I20260812 02:49:02.344991 11770 raft_consensus.cc:348] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } }
I20260812 02:49:02.345623 11770 raft_consensus.cc:729] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aef9fe4efb6f4b4b9645b5a075ec7de6, State: Initialized, Role: FOLLOWER
I20260812 02:49:02.346330 11770 consensus_queue.cc:260] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } }
I20260812 02:49:02.347922 11983 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.348706 11983 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:02.349298 11770 sys_catalog.cc:564] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:49:02.349408 11738 raft_consensus.cc:482] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 02:49:02.349690 11738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [term 0 FOLLOWER]: Advancing to term 1
W20260812 02:49:02.352923 11715 tablet.cc:2369] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:49:02.354508 11738 raft_consensus.cc:504] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } }
I20260812 02:49:02.357147 11826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "163c0e6dece14590bf2fa25faa4ac869"
I20260812 02:49:02.357734 11826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:49:02.361446 11826 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aef9fe4efb6f4b4b9645b5a075ec7de6 in term 1.
I20260812 02:49:02.362401 11712 leader_election.cc:302] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163c0e6dece14590bf2fa25faa4ac869, aef9fe4efb6f4b4b9645b5a075ec7de6; no voters:
I20260812 02:49:02.364233 11915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8b2f0cdda5c2418082ff3b9f20968c12"
I20260812 02:49:02.364871 11915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:49:02.368739 11915 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aef9fe4efb6f4b4b9645b5a075ec7de6 in term 1.
I20260812 02:49:02.369635 11738 leader_election.cc:288] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [CANDIDATE]: Term 1 election: Requested vote from peers 163c0e6dece14590bf2fa25faa4ac869 (127.17.196.253:41733), 8b2f0cdda5c2418082ff3b9f20968c12 (127.17.196.252:40169)
I20260812 02:49:02.377420 11983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:49:02.387099 11983 raft_consensus.cc:686] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [term 1 LEADER]: Becoming Leader. State: Replica: aef9fe4efb6f4b4b9645b5a075ec7de6, State: Running, Role: LEADER
I20260812 02:49:02.388207 11983 consensus_queue.cc:237] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } }
I20260812 02:49:02.398923 11991 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aef9fe4efb6f4b4b9645b5a075ec7de6. Latest consensus state: current_term: 1 leader_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.399513 11991 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: This master's current role is: LEADER
I20260812 02:49:02.405827 11999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:49:02.421514 11999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 02:49:02.436187 11802 tablet.cc:2369] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:49:02.442929 12008 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 02:49:02.470628 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.429780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38544 (local address 127.17.196.253:41733)
0812 02:49:02.429966 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.429973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.435095 (+ 5122us) server_negotiation.cc:407] Connection header received
0812 02:49:02.441836 (+ 6741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.441843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.441939 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.442024 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.445579 (+ 3555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.446750 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.448716 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.449436 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.449572 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.457985 (+ 8413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.457993 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.457999 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.458064 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.466328 (+ 8264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.467362 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.467368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.467371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.467553 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.468009 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.468014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.468018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.468444 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.468753 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.469865 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.470126 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 02:49:02.481277 11944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.411401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.17.196.252:40169)
0812 02:49:02.411688 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.411696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.418302 (+ 6606us) server_negotiation.cc:407] Connection header received
0812 02:49:02.418726 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.418737 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.418867 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.418994 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.429676 (+ 10682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.431322 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.434926 (+ 3604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.435925 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.436060 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.457921 (+ 21861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.457929 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.457935 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.457984 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.464056 (+ 6072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.467524 (+ 3468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.467530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.467535 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.467714 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.478202 (+ 10488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.478208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.478213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.478694 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.480323 (+ 1629us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.480521 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.480810 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:49:02.495986 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:02.413166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.17.196.254:34373)
0812 02:49:02.413410 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.413421 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.426047 (+ 12626us) server_negotiation.cc:407] Connection header received
0812 02:49:02.426327 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.426333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.426420 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.426595 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.441576 (+ 14981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.443156 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.457710 (+ 14554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:02.482656 (+ 24946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:02.482831 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:02.486873 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:02.486882 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:02.486888 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:02.486956 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:02.491215 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.492230 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.492238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.492242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.492411 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:02.492947 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:02.492953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:02.492957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:02.493416 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:02.493765 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:02.495172 (+ 1407us) server_negotiation.cc:299] Negotiation successful
0812 02:49:02.495506 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 02:49:02.509008 11915 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [term 1 FOLLOWER]: Refusing update from remote peer aef9fe4efb6f4b4b9645b5a075ec7de6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:49:02.510272 11826 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [term 1 FOLLOWER]: Refusing update from remote peer aef9fe4efb6f4b4b9645b5a075ec7de6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:49:02.511297 11983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:49:02.520092 11983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:49:02.537225 11965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aef9fe4efb6f4b4b9645b5a075ec7de6. Latest consensus state: current_term: 1 leader_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.537590 11965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:02.545869 12009 mvcc.cc:205] Tried to move back new op lower bound from 7317516052501295104 to 7317516052041285632. Current Snapshot: MvccSnapshot[applied={T|T < 7317516052501295104}]
I20260812 02:49:02.559196 11965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.559518 11965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:02.564615 11991 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.564908 11991 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: This master's current role is: LEADER
I20260812 02:49:02.567090 11949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aef9fe4efb6f4b4b9645b5a075ec7de6. Latest consensus state: current_term: 1 leader_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.567409 11949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:02.569720 11983 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.570188 11983 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6 [sys.catalog]: This master's current role is: LEADER
I20260812 02:49:02.571209 11949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aef9fe4efb6f4b4b9645b5a075ec7de6" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 34373 } } peers { permanent_uuid: "163c0e6dece14590bf2fa25faa4ac869" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 41733 } } peers { permanent_uuid: "8b2f0cdda5c2418082ff3b9f20968c12" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 40169 } } }
I20260812 02:49:02.571447 11949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:02.577986 11999 catalog_manager.cc:1347] Generated new cluster ID: 1c4ac2330f7a442bbbb8a7963d6ebc22
I20260812 02:49:02.578105 11999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:49:02.621767 11999 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:49:02.623836 11999 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:49:02.652746 11999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aef9fe4efb6f4b4b9645b5a075ec7de6: Generated new TSK 0
I20260812 02:49:02.653975 11999 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:49:02.811044 11945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.801079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.17.196.254:34373)
0812 02:49:02.801368 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.801374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.808279 (+ 6905us) server_negotiation.cc:407] Connection header received
0812 02:49:02.808429 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.808434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.808514 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.808615 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.810731 (+ 2116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60246: BlockingRecv error: recv got EOF from 127.0.0.1:60246 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 02:49:02.813923 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.808960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38554 (local address 127.17.196.253:41733)
0812 02:49:02.809169 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.809175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.811982 (+ 2807us) server_negotiation.cc:407] Connection header received
0812 02:49:02.812366 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.812372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.812474 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.812547 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.813657 (+ 1110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38554: BlockingRecv error: recv got EOF from 127.0.0.1:38554 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 02:49:02.820173 11944 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.797409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44588 (local address 127.17.196.252:40169)
0812 02:49:02.797675 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.797683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.807618 (+ 9935us) server_negotiation.cc:407] Connection header received
0812 02:49:02.807848 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.807855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.807961 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.808106 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.819875 (+ 11769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44588: BlockingRecv error: recv got EOF from 127.0.0.1:44588 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260812 02:49:02.862668 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.851378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.17.196.253:41733)
0812 02:49:02.851635 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.851642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.859522 (+ 7880us) server_negotiation.cc:407] Connection header received
0812 02:49:02.860154 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.860161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.860303 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.860412 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.862371 (+ 1959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38556: BlockingRecv error: recv got EOF from 127.0.0.1:38556 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260812 02:49:02.894372 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.891489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38558 (local address 127.17.196.253:41733)
0812 02:49:02.891785 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.891797 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.892520 (+ 723us) server_negotiation.cc:407] Connection header received
0812 02:49:02.892794 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.892801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.892888 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.893029 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.894090 (+ 1061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38558: BlockingRecv error: recv got EOF from 127.0.0.1:38558 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260812 02:49:02.902616 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.900066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38570 (local address 127.17.196.253:41733)
0812 02:49:02.900302 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.900313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.900958 (+ 645us) server_negotiation.cc:407] Connection header received
0812 02:49:02.901218 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.901223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.901303 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.901425 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.902330 (+ 905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38570: BlockingRecv error: recv got EOF from 127.0.0.1:38570 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 02:49:02.913537 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.911023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38574 (local address 127.17.196.253:41733)
0812 02:49:02.911341 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.911349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.911643 (+ 294us) server_negotiation.cc:407] Connection header received
0812 02:49:02.911835 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.911842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.911951 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.912105 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.913226 (+ 1121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38574: BlockingRecv error: recv got EOF from 127.0.0.1:38574 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
W20260812 02:49:02.928323 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.925920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.17.196.253:41733)
0812 02:49:02.926145 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.926153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.926568 (+ 415us) server_negotiation.cc:407] Connection header received
0812 02:49:02.926762 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.926772 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.926914 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.927054 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.928031 (+ 977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38578: BlockingRecv error: recv got EOF from 127.0.0.1:38578 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260812 02:49:02.951185 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.948494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38582 (local address 127.17.196.253:41733)
0812 02:49:02.948727 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.948734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.949106 (+ 372us) server_negotiation.cc:407] Connection header received
0812 02:49:02.949469 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.949473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.949626 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.949722 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.950892 (+ 1170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38582: BlockingRecv error: recv got EOF from 127.0.0.1:38582 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260812 02:49:02.991019 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:02.988545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38592 (local address 127.17.196.253:41733)
0812 02:49:02.988767 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:02.988775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:02.989377 (+ 602us) server_negotiation.cc:407] Connection header received
0812 02:49:02.989682 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:02.989687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:02.989787 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:02.989908 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:02.990755 (+ 847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38592: BlockingRecv error: recv got EOF from 127.0.0.1:38592 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 02:49:03.062422 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:03.059433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.17.196.253:41733)
0812 02:49:03.059747 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:03.059756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:03.060124 (+ 368us) server_negotiation.cc:407] Connection header received
0812 02:49:03.060618 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:03.060624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:03.060719 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:03.060867 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:03.062153 (+ 1286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38604: BlockingRecv error: recv got EOF from 127.0.0.1:38604 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:49:03.187846 11962 catalog_manager.cc:1259] Loaded cluster ID: 1c4ac2330f7a442bbbb8a7963d6ebc22
I20260812 02:49:03.187954 11962 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: loading cluster ID for follower catalog manager: success
I20260812 02:49:03.190626 11962 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: acquiring CA information for follower catalog manager: success
I20260812 02:49:03.192488 11962 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8b2f0cdda5c2418082ff3b9f20968c12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 02:49:03.196419 11947 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:03.194384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.17.196.253:41733)
0812 02:49:03.194629 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:03.194637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:03.194966 (+ 329us) server_negotiation.cc:407] Connection header received
0812 02:49:03.195235 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:03.195240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:03.195322 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:03.195438 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:03.196218 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38618: BlockingRecv error: recv got EOF from 127.0.0.1:38618 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:49:03.242188 11978 catalog_manager.cc:1259] Loaded cluster ID: 1c4ac2330f7a442bbbb8a7963d6ebc22
I20260812 02:49:03.242300 11978 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: loading cluster ID for follower catalog manager: success
I20260812 02:49:03.244860 11978 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: acquiring CA information for follower catalog manager: success
I20260812 02:49:03.246929 11978 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 163c0e6dece14590bf2fa25faa4ac869: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:49:03.471907 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:03.464766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.17.196.253:41733)
0812 02:49:03.464953 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:03.464958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:03.465366 (+ 408us) server_negotiation.cc:407] Connection header received
0812 02:49:03.465588 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:03.465594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:03.465806 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:03.465953 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:03.466498 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.467666 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.469325 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.469886 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.470017 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:03.470655 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:03.471257 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:03.471388 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:03.471605 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 02:49:03.471689 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 02:49:03.490651 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:03.484047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.17.196.252:40169)
0812 02:49:03.484592 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:03.484597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:03.484729 (+ 132us) server_negotiation.cc:407] Connection header received
0812 02:49:03.484939 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:03.484944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:03.485105 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:03.485265 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:03.485777 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.486579 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.488186 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.488600 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.488697 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:03.489204 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:03.489882 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:03.489995 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:03.490180 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 02:49:03.490254 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":267,"threads_started":1}
I20260812 02:49:03.506992 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:03.498619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.17.196.252:40169)
0812 02:49:03.498864 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:03.498874 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:03.500967 (+ 2093us) server_negotiation.cc:407] Connection header received
0812 02:49:03.501127 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:03.501137 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:03.501247 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:03.501332 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:03.502099 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.503069 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.504834 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.505457 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.505655 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:03.506154 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:03.506296 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:03.506373 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:03.506568 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 02:49:03.506662 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:49:03.507705 12102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:03.498351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.17.196.254:34373)
0812 02:49:03.499382 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:03.499392 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:03.499836 (+ 444us) server_negotiation.cc:407] Connection header received
0812 02:49:03.500160 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:03.500168 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:03.500411 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:03.500626 (+ 215us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:03.501228 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.502325 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.504094 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.504627 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.504760 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:03.505410 (+ 650us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:03.505920 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:03.506077 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:03.507281 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0812 02:49:03.507387 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":676,"threads_started":1}
I20260812 02:49:03.509034 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:03.499943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.17.196.253:41733)
0812 02:49:03.500150 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:03.500156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:03.500902 (+ 746us) server_negotiation.cc:407] Connection header received
0812 02:49:03.501566 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:03.501571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:03.501644 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:03.501757 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:03.503231 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.504068 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.506617 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:03.507398 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:03.507547 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:03.508285 (+ 738us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:03.508446 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:03.508542 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:03.508729 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 02:49:03.508818 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:49:03.514163 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11691
I20260812 02:49:03.515501 11765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:49:03.720220 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 11691
W20260812 02:49:03.790005 11885 connection.cc:535] server connection from 127.0.0.1:44564 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 02:49:03.790051 11797 connection.cc:535] server connection from 127.0.0.1:40042 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:49:03.792143 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11778
I20260812 02:49:03.794837 11849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:49:04.035907 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 11778
I20260812 02:49:04.082911 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 11864
I20260812 02:49:04.084204 11936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:49:04.194715 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 11864
2026-08-12T02:49:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5509 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:49:04 dist-test-slave-ss4n krb5kdc[12113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:49:04 dist-test-slave-ss4n krb5kdc[12113](info): set up 2 sockets
Aug 12 02:49:04 dist-test-slave-ss4n krb5kdc[12113](info): commencing operation
krb5kdc: starting...
W20260812 02:49:05.556649 18195 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.001s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/test-user.keytab.
Aug 12 02:49:05 dist-test-slave-ss4n krb5kdc[12113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:49:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
Entry for principal kudu/127.17.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab.
I20260812 02:49:05.945670 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44969
--webserver_interface=127.17.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44969,127.17.196.253:39677,127.17.196.252:33915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:49:06.313678 12130 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:49:06.314054 12130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:49:06.314122 12130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:49:06.314206 12130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:49:06.321870 12130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:49:06.321981 12130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:49:06.322041 12130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:49:06.322094 12130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:49:06.322183 12130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:49:06.332687 12130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44969,127.17.196.253:39677,127.17.196.252:33915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.254.keytab
--principal=kudu/127.17.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.254:44969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:49:06.334169 12130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:49:06.335566 12130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:49:06.348640 12140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:06.348573 12138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:06.348789 12137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:49:06.349424 12130 server_base.cc:1034] running on GCE node
I20260812 02:49:06.350423 12130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:49:06.352147 12130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:49:06.353447 12130 hybrid_clock.cc:648] HybridClock initialized: now 1786502946353327 us; error 88 us; skew 500 ppm
I20260812 02:49:06.353952 12130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:49:06 dist-test-slave-ss4n krb5kdc[12113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502946, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:49:06.357849 12130 init.cc:377] Logged in from keytab as kudu/127.17.196.254@KRBTEST.COM (short username kudu)
I20260812 02:49:06.359539 12130 webserver.cc:460] Webserver started at http://127.17.196.254:36881/ using document root <none> and password file <none>
I20260812 02:49:06.360334 12130 fs_manager.cc:359] Metadata directory not provided
I20260812 02:49:06.360442 12130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:49:06.360759 12130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:49:06.364472 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/data/instance:
uuid: "08c511f8f7b4464a86f84106c629ef83"
format_stamp: "Formatted at 2026-08-12 02:49:06 on dist-test-slave-ss4n"
server_key: "b4405948961c75ac618fd7d0e3ad982f"
server_key_iv: "75453f1e1251589fcfedc5eba59b1986"
server_key_version: "encryptionkey@0"
I20260812 02:49:06.365356 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/wal/instance:
uuid: "08c511f8f7b4464a86f84106c629ef83"
format_stamp: "Formatted at 2026-08-12 02:49:06 on dist-test-slave-ss4n"
server_key: "b4405948961c75ac618fd7d0e3ad982f"
server_key_iv: "75453f1e1251589fcfedc5eba59b1986"
server_key_version: "encryptionkey@0"
I20260812 02:49:06.371402 12130 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:49:06.375312 12148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:49:06.376945 12130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:49:06.377136 12130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/wal
uuid: "08c511f8f7b4464a86f84106c629ef83"
format_stamp: "Formatted at 2026-08-12 02:49:06 on dist-test-slave-ss4n"
server_key: "b4405948961c75ac618fd7d0e3ad982f"
server_key_iv: "75453f1e1251589fcfedc5eba59b1986"
server_key_version: "encryptionkey@0"
I20260812 02:49:06.377460 12130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:49:06.409118 12130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:49:06.414829 12130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:49:06.415185 12130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:49:06.440572 12130 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.254:44969
I20260812 02:49:06.440604 12211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.254:44969 every 8 connection(s)
I20260812 02:49:06.442217 12130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/data/info.pb
I20260812 02:49:06.443040 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 12130
I20260812 02:49:06.443259 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-0/wal/instance
I20260812 02:49:06.443812 18195 external_mini_cluster.cc:1428] Setting key 9e6a7362bc365f864ba5fdfac987b205
I20260812 02:49:06.448464 12212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:49:06.456079 12212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:06 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502946, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.196.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.253@KRBTEST.COM" created.
I20260812 02:49:06.483327 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.459697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44969 (local address 127.0.0.1:53534)
0812 02:49:06.460646 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.460700 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:06.462959 (+ 2259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:06.464011 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:06.464018 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:06.464811 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:06.465179 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:06.465198 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:06.466956 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:06.466961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:06.468079 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:06.468091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:06.469083 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:06.469091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:06.469278 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:06.470211 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:06.470240 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:06.473231 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:06.477661 (+ 4430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:06.477677 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:06.477692 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:06.478341 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:06.478916 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:06.478923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:06.478927 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:06.479240 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:06.480226 (+ 986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:06.480236 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:06.480539 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 02:49:06.480881 (+ 342us) client_negotiation.cc:240] Negotiation successful
0812 02:49:06.481529 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":421,"threads_started":1}
I20260812 02:49:06.483351 12218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.459698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.17.196.254:44969)
0812 02:49:06.461574 (+ 1876us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:06.461585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:06.461645 (+ 60us) server_negotiation.cc:407] Connection header received
0812 02:49:06.463438 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:06.463449 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:06.463769 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:06.464004 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:06.465415 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:06.466794 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:06.468233 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:06.468829 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:06.469047 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:06.473566 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:06.473577 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:06.473593 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:06.473632 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:06.477384 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:06.478549 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:06.478558 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:06.478561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:06.478757 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:06.479403 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:06.479409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:06.479415 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:06.480042 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:06.480368 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:06.481104 (+ 736us) server_negotiation.cc:299] Negotiation successful
0812 02:49:06.481730 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":265,"server-negotiator.queue_time_us":1631,"thread_start_us":1407,"threads_started":1}
I20260812 02:49:06.487144 12212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:49:06.488511 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.488039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33782)
0812 02:49:06.488228 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.488370 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 02:49:06.488830 12149 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
I20260812 02:49:06.490558 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.490171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33784)
0812 02:49:06.490335 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.490415 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 02:49:06.490782 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:39677: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab.
Entry for principal kudu/127.17.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab.
I20260812 02:49:06.526809 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.253:39677
--webserver_interface=127.17.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab
--principal=kudu/127.17.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44969,127.17.196.253:39677,127.17.196.252:33915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:49:06.540155 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } attempt: 1
I20260812 02:49:06.541810 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.541133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33796)
0812 02:49:06.541403 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.541617 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 02:49:06.543220 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.542699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33808)
0812 02:49:06.542933 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.543038 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 02:49:06.543617 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:39677: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
I20260812 02:49:06.611935 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } attempt: 2
I20260812 02:49:06.613370 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.612767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33814)
0812 02:49:06.613070 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.613191 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 02:49:06.614635 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.614179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33828)
0812 02:49:06.614349 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.614456 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 02:49:06.614961 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:39677: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
I20260812 02:49:06.706305 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } attempt: 3
I20260812 02:49:06.707931 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.707172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33840)
0812 02:49:06.707501 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.707672 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260812 02:49:06.709417 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.708775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33856)
0812 02:49:06.709028 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.709164 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 02:49:06.709731 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:39677: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
I20260812 02:49:06.853052 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } attempt: 4
I20260812 02:49:06.854637 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.853890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33866)
0812 02:49:06.854294 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.854426 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260812 02:49:06.855822 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:06.855354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33882)
0812 02:49:06.855544 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:06.855653 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 02:49:06.856045 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.253:39677: Network error: Client connection negotiation failed: client connection to 127.17.196.253:39677: connect: Connection refused (error 111)
W20260812 02:49:06.909857 12222 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:49:06.910238 12222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:49:06.910337 12222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:49:06.910516 12222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:49:06.918327 12222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:49:06.918429 12222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:49:06.918536 12222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:49:06.918624 12222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:49:06.918721 12222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:49:06.929112 12222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44969,127.17.196.253:39677,127.17.196.252:33915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.253.keytab
--principal=kudu/127.17.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.253:39677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:49:06.930586 12222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:49:06.931957 12222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:49:06.944610 12229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:06.944587 12228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:06.944751 12232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:49:06.945746 12222 server_base.cc:1034] running on GCE node
I20260812 02:49:06.946655 12222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:49:06.948143 12222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:49:06.949432 12222 hybrid_clock.cc:648] HybridClock initialized: now 1786502946949344 us; error 61 us; skew 500 ppm
I20260812 02:49:06.949888 12222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:49:06 dist-test-slave-ss4n krb5kdc[12113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502946, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:49:06.953775 12222 init.cc:377] Logged in from keytab as kudu/127.17.196.253@KRBTEST.COM (short username kudu)
I20260812 02:49:06.955675 12222 webserver.cc:460] Webserver started at http://127.17.196.253:36791/ using document root <none> and password file <none>
I20260812 02:49:06.956548 12222 fs_manager.cc:359] Metadata directory not provided
I20260812 02:49:06.956714 12222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:49:06.957170 12222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:49:06.960908 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/data/instance:
uuid: "2b863f588de742068ac2c92031de2a00"
format_stamp: "Formatted at 2026-08-12 02:49:06 on dist-test-slave-ss4n"
server_key: "60aaa82ccbaf32888bed3695db056f38"
server_key_iv: "9b7ea4863c7c5910601156b606ccbdb1"
server_key_version: "encryptionkey@0"
I20260812 02:49:06.962005 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/wal/instance:
uuid: "2b863f588de742068ac2c92031de2a00"
format_stamp: "Formatted at 2026-08-12 02:49:06 on dist-test-slave-ss4n"
server_key: "60aaa82ccbaf32888bed3695db056f38"
server_key_iv: "9b7ea4863c7c5910601156b606ccbdb1"
server_key_version: "encryptionkey@0"
I20260812 02:49:06.968694 12222 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 02:49:06.973042 12240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:49:06.974783 12222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:49:06.975036 12222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/wal
uuid: "2b863f588de742068ac2c92031de2a00"
format_stamp: "Formatted at 2026-08-12 02:49:06 on dist-test-slave-ss4n"
server_key: "60aaa82ccbaf32888bed3695db056f38"
server_key_iv: "9b7ea4863c7c5910601156b606ccbdb1"
server_key_version: "encryptionkey@0"
I20260812 02:49:06.975320 12222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:49:06.998832 12222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:49:07.003921 12222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:49:07.004244 12222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:49:07.025961 12222 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.253:39677
I20260812 02:49:07.025976 12294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.253:39677 every 8 connection(s)
I20260812 02:49:07.027407 12222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/data/info.pb
I20260812 02:49:07.031390 12295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:49:07.035598 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 12222
I20260812 02:49:07.035815 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-1/wal/instance
I20260812 02:49:07.036305 18195 external_mini_cluster.cc:1428] Setting key 4a808206e18518a2a1c71cbff12f4512
I20260812 02:49:07.038389 12295 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502946, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:49:07.063195 12298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.041133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.17.196.254:44969)
0812 02:49:07.041797 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.041807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.043076 (+ 1269us) server_negotiation.cc:407] Connection header received
0812 02:49:07.044865 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.044872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.044974 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.045115 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.046984 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.047712 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.049280 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.049785 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.049928 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.054898 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.054905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.054908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.054968 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.058395 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.059829 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.059834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.059837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.059990 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.060858 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.060863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.060865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.061208 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.061350 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.062643 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.062865 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":359,"threads_started":1}
I20260812 02:49:07.065081 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.041807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44969 (local address 127.0.0.1:53550)
0812 02:49:07.042681 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.042737 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:07.044560 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:07.045386 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:07.045399 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:07.046025 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:07.046741 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.046772 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.047888 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.047895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.049116 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.049135 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.050038 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.050046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.050293 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.051067 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:07.051105 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:07.054555 (+ 3450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:07.058618 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.058634 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.058659 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.059625 (+ 966us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.060343 (+ 718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.060353 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.060360 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.060673 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.061389 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:07.061402 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:07.062059 (+ 657us) client_negotiation.cc:769] Sending connection context
0812 02:49:07.062476 (+ 417us) client_negotiation.cc:240] Negotiation successful
0812 02:49:07.063227 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":364,"threads_started":1}
I20260812 02:49:07.067601 12295 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.196.252@KRBTEST.COM" created.
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502946, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:49:07.093807 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.068733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33886)
0812 02:49:07.068961 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.069007 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:07.069152 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:07.075271 (+ 6119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:07.075278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:07.075732 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:07.075899 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.075909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.078315 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.078321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.080097 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.080120 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.081611 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.081617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.081878 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.082705 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:07.082726 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:07.085686 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:07.090826 (+ 5140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.090838 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.090841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.091310 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.091831 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.091835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.091838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.091997 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.092884 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:07.092889 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:07.093040 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 02:49:07.093162 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 02:49:07.093406 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":53}
I20260812 02:49:07.094329 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.068892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33886 (local address 127.17.196.253:39677)
0812 02:49:07.074596 (+ 5704us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.074604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.074646 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:49:07.074716 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.074723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.074975 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.075214 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.076154 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.078133 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.080295 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.081378 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.081655 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.085967 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.085996 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.086020 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.086074 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.090644 (+ 4570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.091469 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.091483 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.091490 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.091668 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.092119 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.092128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.092134 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.092697 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.092940 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.093362 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.093709 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":5478,"thread_start_us":5281,"threads_started":1}
I20260812 02:49:07.097122 12295 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:49:07.098384 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.097894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35434)
0812 02:49:07.098128 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.098233 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260812 02:49:07.098670 12242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.100347 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.099982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35436)
0812 02:49:07.100168 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.100244 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 02:49:07.100608 12295 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab.
Entry for principal kudu/127.17.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab.
I20260812 02:49:07.122750 18195 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
/tmp/dist-test-taskDYHVZq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.252:33915
--webserver_interface=127.17.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab
--principal=kudu/127.17.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.196.212:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.196.254:44969,127.17.196.253:39677,127.17.196.252:33915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:49:07.149926 12295 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 1
I20260812 02:49:07.151284 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.150800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35444)
0812 02:49:07.151009 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.151123 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 02:49:07.152393 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.152019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35456)
0812 02:49:07.152202 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.152291 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 02:49:07.152622 12295 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.155313 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } attempt: 5
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502946, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:49:07.169636 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.156135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.17.196.253:39677)
0812 02:49:07.156301 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.156307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.156404 (+ 97us) server_negotiation.cc:407] Connection header received
0812 02:49:07.156708 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.156713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.156790 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.156899 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.157764 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.158497 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.159662 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.160144 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.160281 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.163626 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.163633 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.163641 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.163685 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.166707 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.167446 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.167455 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.167461 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.167622 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.168040 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.168046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.168050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.168361 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.168533 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.168976 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.169229 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:49:07.169651 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.156061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33900)
0812 02:49:07.156275 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.156301 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:07.156539 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:07.156934 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:07.156940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:07.157365 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:07.157620 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.157630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.158636 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.158642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.159520 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.159531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.160297 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.160302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.160570 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.161289 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:07.161308 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:07.163416 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:07.166874 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.166880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.166885 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.167306 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.167747 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.167752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.167757 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.167912 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.168501 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:07.168507 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:07.168675 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 02:49:07.168846 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 02:49:07.169214 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 02:49:07.170814 12212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:49:07.172053 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.171597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35464)
0812 02:49:07.171798 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.171879 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 02:49:07.173158 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.172775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35474)
0812 02:49:07.172935 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.173029 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 02:49:07.173574 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.220885 12295 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 2
I20260812 02:49:07.222378 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.221768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35488)
0812 02:49:07.222075 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.222198 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260812 02:49:07.223798 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.223331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35504)
0812 02:49:07.223592 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.223676 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260812 02:49:07.224097 12295 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.224849 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 1
I20260812 02:49:07.226316 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.225727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35510)
0812 02:49:07.226039 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.226150 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260812 02:49:07.227365 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.227011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35524)
0812 02:49:07.227171 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.227254 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 02:49:07.227595 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.295820 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 2
I20260812 02:49:07.297065 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.296577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35530)
0812 02:49:07.296808 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.296897 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 02:49:07.298264 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.297889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35536)
0812 02:49:07.298043 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.298130 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 02:49:07.298537 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.315351 12295 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 3
I20260812 02:49:07.316578 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.316069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35550)
0812 02:49:07.316294 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.316400 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 02:49:07.317818 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.317334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35556)
0812 02:49:07.317612 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.317698 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260812 02:49:07.318032 12295 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.404801 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 3
I20260812 02:49:07.406246 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.405631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35558)
0812 02:49:07.405849 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.405951 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 02:49:07.407449 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.407015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35574)
0812 02:49:07.407248 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.407344 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260812 02:49:07.407653 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.461282 12295 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 4
I20260812 02:49:07.462809 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.462222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35576)
0812 02:49:07.462553 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.462647 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 02:49:07.463928 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.463582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35588)
0812 02:49:07.463745 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.463816 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 02:49:07.464134 12295 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
W20260812 02:49:07.513051 12306 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:49:07.513485 12306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:49:07.513640 12306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:49:07.513782 12306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:49:07.521345 12306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:49:07.521492 12306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:49:07.521637 12306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:49:07.521728 12306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:49:07.521821 12306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:49:07.532169 12306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.196.212:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.196.254:44969,127.17.196.253:39677,127.17.196.252:33915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/krb5kdc/kudu_127.17.196.252.keytab
--principal=kudu/127.17.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.196.252:33915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:49:07.533612 12306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:49:07.534992 12306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:49:07.547320 12314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:07.547415 12315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:49:07.547420 12317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:49:07.548741 12306 server_base.cc:1034] running on GCE node
I20260812 02:49:07.549662 12306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:49:07.551237 12306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:49:07.552520 12306 hybrid_clock.cc:648] HybridClock initialized: now 1786502947552436 us; error 61 us; skew 500 ppm
I20260812 02:49:07.552902 12306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502947, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:49:07.556416 12306 init.cc:377] Logged in from keytab as kudu/127.17.196.252@KRBTEST.COM (short username kudu)
I20260812 02:49:07.558279 12306 webserver.cc:460] Webserver started at http://127.17.196.252:40373/ using document root <none> and password file <none>
I20260812 02:49:07.559053 12306 fs_manager.cc:359] Metadata directory not provided
I20260812 02:49:07.559197 12306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:49:07.559588 12306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:49:07.562846 12306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/data/instance:
uuid: "3db5dc9ed6fc451984e9fc01bce3af3d"
format_stamp: "Formatted at 2026-08-12 02:49:07 on dist-test-slave-ss4n"
server_key: "d2322d4b69d2547f96054b4129dc0e28"
server_key_iv: "09dc9bbad04538962e30b24428437e5d"
server_key_version: "encryptionkey@0"
I20260812 02:49:07.563736 12306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/wal/instance:
uuid: "3db5dc9ed6fc451984e9fc01bce3af3d"
format_stamp: "Formatted at 2026-08-12 02:49:07 on dist-test-slave-ss4n"
server_key: "d2322d4b69d2547f96054b4129dc0e28"
server_key_iv: "09dc9bbad04538962e30b24428437e5d"
server_key_version: "encryptionkey@0"
I20260812 02:49:07.569471 12306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:49:07.573331 12326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:49:07.574806 12306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:49:07.575040 12306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/data,/tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/wal
uuid: "3db5dc9ed6fc451984e9fc01bce3af3d"
format_stamp: "Formatted at 2026-08-12 02:49:07 on dist-test-slave-ss4n"
server_key: "d2322d4b69d2547f96054b4129dc0e28"
server_key_iv: "09dc9bbad04538962e30b24428437e5d"
server_key_version: "encryptionkey@0"
I20260812 02:49:07.575285 12306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:49:07.584856 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 4
I20260812 02:49:07.586061 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.585613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35600)
0812 02:49:07.585807 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.585907 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 02:49:07.587208 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.586819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35614)
0812 02:49:07.586999 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.587075 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 02:49:07.587459 12212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.196.252:33915: Network error: Client connection negotiation failed: client connection to 127.17.196.252:33915: connect: Connection refused (error 111)
I20260812 02:49:07.600457 12306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:49:07.605744 12306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:49:07.606069 12306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:49:07.627694 12306 rpc_server.cc:307] RPC server started. Bound to: 127.17.196.252:33915
I20260812 02:49:07.627737 12388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.196.252:33915 every 8 connection(s)
I20260812 02:49:07.629243 12306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/data/info.pb
I20260812 02:49:07.630975 18195 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu as pid 12306
I20260812 02:49:07.631212 18195 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDYHVZq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502547576936-18195-0/minicluster-data/master-2/wal/instance
I20260812 02:49:07.631917 18195 external_mini_cluster.cc:1428] Setting key f818076143f87e55bc2f616b03f62402
I20260812 02:49:07.634575 12389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:49:07.644420 12389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502947, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
I20260812 02:49:07.659391 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.634026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.17.196.252:33915)
0812 02:49:07.635011 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.635024 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.635070 (+ 46us) server_negotiation.cc:407] Connection header received
0812 02:49:07.636954 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.636979 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.637428 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.637961 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.640127 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.641839 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.643938 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.644795 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.644956 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.650729 (+ 5773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.650739 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.650750 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.650800 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.654338 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.655128 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.655146 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.655154 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.655344 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.655823 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.655829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.655835 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.656449 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.656791 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.657315 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.657841 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":463,"threads_started":1}
I20260812 02:49:07.665408 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.648230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.254:44969 (local address 127.0.0.1:53556)
0812 02:49:07.649208 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.649270 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:07.649511 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:07.650004 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:07.650012 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:07.650718 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:07.651048 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.651062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.652308 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.652313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.653668 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.653696 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.654544 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.654550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.654765 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.655479 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:07.655501 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:07.658007 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:07.662023 (+ 4016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.662038 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.662047 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.662715 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.663227 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.663233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.663236 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.663528 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.664383 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:07.664390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:07.664643 (+ 253us) client_negotiation.cc:769] Sending connection context
0812 02:49:07.664810 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 02:49:07.665111 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":490,"threads_started":1}
I20260812 02:49:07.666118 12394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.648124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.17.196.254:44969)
0812 02:49:07.648771 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.648778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.649400 (+ 622us) server_negotiation.cc:407] Connection header received
0812 02:49:07.649704 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.649708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.649780 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.649918 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.651277 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.652086 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.653844 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.654315 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.654471 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.658239 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.658247 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.658253 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.658312 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.661859 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.662871 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.662880 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.662883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.663080 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.663686 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.663695 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.663702 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.664207 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.664448 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.664996 (+ 548us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.665335 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":286,"threads_started":1}
I20260812 02:49:07.667780 12389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.254@KRBTEST.COM
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502947, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:49:07.684592 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.663014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.17.196.254:44969)
0812 02:49:07.666329 (+ 3315us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.666336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.666384 (+ 48us) server_negotiation.cc:407] Connection header received
0812 02:49:07.666445 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.666451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.666580 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.666707 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.668390 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.669815 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.671280 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.671758 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.671863 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.677346 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.677357 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.677366 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.677419 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.680832 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.681840 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.681847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.681851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.682025 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.682669 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.682673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.682675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.683167 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.683478 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.683843 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.684110 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3153,"thread_start_us":2928,"threads_started":1}
I20260812 02:49:07.684876 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.668734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.253:39677 (local address 127.0.0.1:33910)
0812 02:49:07.668877 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.668893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:07.668972 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:07.669430 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:07.669433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:07.669930 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:07.670136 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.670148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.671192 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.671200 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.672313 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.672329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.673274 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.673281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.673557 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.674107 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:07.674131 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:07.677274 (+ 3143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:07.681574 (+ 4300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.681578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.681581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.682258 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.682774 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.682784 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.682791 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.682972 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.683785 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:07.683791 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:07.683982 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 02:49:07.684153 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 02:49:07.684464 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 02:49:07.685016 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.668694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33910 (local address 127.17.196.253:39677)
0812 02:49:07.669086 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.669093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.669116 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:49:07.669194 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.669200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.669288 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.669400 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.670310 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.671033 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.672488 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.673098 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.673208 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.677562 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.677569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.677572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.677611 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.681350 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.682416 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.682425 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.682429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.682629 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.683100 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.683104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.683106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.683650 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.683807 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.684312 (+ 505us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.684578 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242}
I20260812 02:49:07.686478 12389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502947, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.252@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.196.253@KRBTEST.COM
I20260812 02:49:07.701844 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.687533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.17.196.252:33915)
0812 02:49:07.687700 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.687707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.687725 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:49:07.687834 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.687839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.687907 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.688011 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.688769 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.689628 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.690897 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.691380 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.691519 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.695008 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.695015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.695020 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.695065 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.698373 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.699360 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.699364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.699367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.699586 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.700096 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.700100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.700102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.700403 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.700654 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.701098 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.701362 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":71}
I20260812 02:49:07.701902 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.687374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35630)
0812 02:49:07.687584 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.687606 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:07.687716 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:07.688030 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:07.688034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:07.688478 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:07.688622 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.688632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.689759 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.689768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.690745 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.690758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.691564 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.691570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.691798 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.692500 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:07.692532 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:07.694816 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:07.698612 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.698618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.698623 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.699209 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.699735 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.699744 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.699750 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.699943 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.700610 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:07.700615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:07.700778 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 02:49:07.700999 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 02:49:07.701551 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":182}
I20260812 02:49:07.703506 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.686616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.17.196.253:39677)
0812 02:49:07.686812 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.686819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.686927 (+ 108us) server_negotiation.cc:407] Connection header received
0812 02:49:07.687201 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.687207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.687413 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.687525 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.689016 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.689926 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.691314 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.691789 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.691944 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.696862 (+ 4918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.696870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.696877 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.696921 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.700289 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.701032 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.701040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.701043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.701244 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.701771 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.701777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.701783 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.702234 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.702537 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.702868 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.703152 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 02:49:07.708081 12389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: Bootstrap starting.
I20260812 02:49:07.709527 18195 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:49:07.712635 12389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: Neither blocks nor log segments found. Creating new log.
I20260812 02:49:07.714246 12389 log.cc:824] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:49:07.712175 12177 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "08c511f8f7b4464a86f84106c629ef83"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53568
I20260812 02:49:07.718127 12389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: No bootstrap required, opened a new log
I20260812 02:49:07.730858 12389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } }
I20260812 02:49:07.731393 12389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3db5dc9ed6fc451984e9fc01bce3af3d, State: Initialized, Role: FOLLOWER
I20260812 02:49:07.731956 12389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } }
I20260812 02:49:07.733076 12398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:07.733525 12398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:07.734565 12389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [sys.catalog]: configured and running, proceeding with master startup.
W20260812 02:49:07.745851 12410 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:49:07.746100 12410 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:49:07.743085 12306 master_runner.cc:186] Error getting master registration for 127.17.196.254:44969: OK, instance_id { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" instance_seqno: 1786502946432415 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:49:07.756693 12306 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:49:07.763312 12295 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 5
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502946, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.253@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
I20260812 02:49:07.782605 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.764180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35646)
0812 02:49:07.764432 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.764457 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:07.764618 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:07.765098 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:07.765105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:07.765753 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:07.765992 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.766006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.767739 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.767746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.769397 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.769419 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.770678 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.770686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.770922 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.771748 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:07.771780 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:07.775260 (+ 3480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:07.779096 (+ 3836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.779102 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.779107 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.779790 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.780297 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.780303 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.780307 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.780552 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.781221 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:07.781226 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:07.781458 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 02:49:07.781766 (+ 308us) client_negotiation.cc:240] Negotiation successful
0812 02:49:07.782175 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 02:49:07.782684 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.764287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.17.196.252:33915)
0812 02:49:07.764515 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.764520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.764541 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:49:07.764798 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.764805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.764911 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.765035 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.766183 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.767554 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.769622 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.770479 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.770624 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.775516 (+ 4892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.775522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.775526 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.775566 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.778900 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.779948 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.779953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.779955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.780101 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.780690 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.780695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.780697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.781065 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.781276 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.781901 (+ 625us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.782167 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:49:07.788409 12295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: Bootstrap starting.
I20260812 02:49:07.792008 12295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: Neither blocks nor log segments found. Creating new log.
I20260812 02:49:07.793162 12295 log.cc:824] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: Log is configured to *not* fsync() on all Append() calls
I20260812 02:49:07.796288 12295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: No bootstrap required, opened a new log
I20260812 02:49:07.804692 12295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } }
I20260812 02:49:07.805125 12295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b863f588de742068ac2c92031de2a00, State: Initialized, Role: FOLLOWER
I20260812 02:49:07.805754 12295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } }
I20260812 02:49:07.806911 12413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:07.807364 12413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:07.808255 12295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 02:49:07.826146 12428 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:49:07.826331 12428 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:49:07.823953 12222 master_runner.cc:186] Error getting master registration for 127.17.196.254:44969: OK, instance_id { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" instance_seqno: 1786502946432415 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:49:07.837916 12222 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 02:49:07.858554 12333 tablet.cc:2369] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:49:07.864734 12212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } attempt: 5
Aug 12 02:49:07 dist-test-slave-ss4n krb5kdc[12113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502946, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.196.254@KRBTEST.COM for kudu/127.17.196.252@KRBTEST.COM
I20260812 02:49:07.883127 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.865611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.17.196.252:33915)
0812 02:49:07.865803 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.865810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.865922 (+ 112us) server_negotiation.cc:407] Connection header received
0812 02:49:07.866136 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.866140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.866234 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.866358 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.867395 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.868601 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.870835 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.871587 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.871727 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.876117 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:07.876122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:07.876126 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:07.876161 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:07.879577 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.880705 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.880711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.880715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.880912 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:07.881380 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:07.881385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:07.881389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:07.881826 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:07.881984 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:07.882445 (+ 461us) server_negotiation.cc:299] Negotiation successful
0812 02:49:07.882707 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:49:07.883507 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.865536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.196.252:33915 (local address 127.0.0.1:35658)
0812 02:49:07.865815 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 02:49:07.865838 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:49:07.865993 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:49:07.866397 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:49:07.866402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:49:07.867002 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:49:07.867234 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.867247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.868803 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.868811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.870601 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:49:07.870625 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.871972 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.871980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:49:07.872212 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:07.872968 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:49:07.872995 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:49:07.875880 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:49:07.879804 (+ 3924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.879811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.879860 (+ 49us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.880520 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.881084 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:49:07.881087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:49:07.881090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:49:07.881239 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:49:07.881954 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:49:07.881959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:49:07.882124 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 02:49:07.882291 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 02:49:07.882712 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260812 02:49:07.889798 12212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83: Bootstrap starting.
I20260812 02:49:07.894050 12212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83: Neither blocks nor log segments found. Creating new log.
I20260812 02:49:07.895483 12212 log.cc:824] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83: Log is configured to *not* fsync() on all Append() calls
I20260812 02:49:07.899458 12212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83: No bootstrap required, opened a new log
I20260812 02:49:07.907786 12212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } }
I20260812 02:49:07.908335 12212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c511f8f7b4464a86f84106c629ef83, State: Initialized, Role: FOLLOWER
I20260812 02:49:07.908929 12212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } }
I20260812 02:49:07.910491 12431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:07.911324 12431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:07.911965 12212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:49:07.912048 12177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 02:49:07.912251 12177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:49:07.916154 12177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } }
W20260812 02:49:07.918895 12156 tablet.cc:2369] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:49:07.919272 12269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08c511f8f7b4464a86f84106c629ef83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2b863f588de742068ac2c92031de2a00"
I20260812 02:49:07.919930 12269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:49:07.925097 12177 leader_election.cc:288] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [CANDIDATE]: Term 1 election: Requested vote from peers 2b863f588de742068ac2c92031de2a00 (127.17.196.253:39677), 3db5dc9ed6fc451984e9fc01bce3af3d (127.17.196.252:33915)
I20260812 02:49:07.925870 12359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08c511f8f7b4464a86f84106c629ef83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d"
I20260812 02:49:07.926494 12359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:49:07.924062 12269 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c511f8f7b4464a86f84106c629ef83 in term 1.
I20260812 02:49:07.929265 12149 leader_election.cc:302] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c511f8f7b4464a86f84106c629ef83, 2b863f588de742068ac2c92031de2a00; no voters:
I20260812 02:49:07.930832 12359 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c511f8f7b4464a86f84106c629ef83 in term 1.
I20260812 02:49:07.933707 12431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:49:07.937265 12431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [term 1 LEADER]: Becoming Leader. State: Replica: 08c511f8f7b4464a86f84106c629ef83, State: Running, Role: LEADER
I20260812 02:49:07.938776 12431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } }
I20260812 02:49:07.952435 12434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08c511f8f7b4464a86f84106c629ef83. Latest consensus state: current_term: 1 leader_uuid: "08c511f8f7b4464a86f84106c629ef83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:07.952903 12434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: This master's current role is: LEADER
I20260812 02:49:07.976907 12447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260812 02:49:08.007555 12246 tablet.cc:2369] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:49:08.049276 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.953952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.17.196.253:39677)
0812 02:49:07.954202 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.954209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.962793 (+ 8584us) server_negotiation.cc:407] Connection header received
0812 02:49:07.965563 (+ 2770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.965572 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.965690 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.965790 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.977246 (+ 11456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.978513 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.993092 (+ 14579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.994200 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.994352 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:08.019350 (+ 24998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:08.019363 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:08.019372 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:08.019459 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:08.029710 (+ 10251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:08.045507 (+ 15797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:08.045519 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:08.045525 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:08.045815 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:08.046485 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:08.046493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:08.046499 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:08.047055 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:08.047406 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:08.048202 (+ 796us) server_negotiation.cc:299] Negotiation successful
0812 02:49:08.048755 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:49:08.057942 12447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:49:08.073091 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.963675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.17.196.252:33915)
0812 02:49:07.963928 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.963937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.972244 (+ 8307us) server_negotiation.cc:407] Connection header received
0812 02:49:07.972542 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.972551 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.972687 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.972782 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.985918 (+ 13136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.987425 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:08.002109 (+ 14684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:08.003249 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:08.003404 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:08.061810 (+ 58406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:08.061822 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:08.061830 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:08.061902 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:08.067675 (+ 5773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:08.069080 (+ 1405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:08.069091 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:08.069097 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:08.069321 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:08.069983 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:08.069992 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:08.069998 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:08.070535 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:08.070892 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:08.071718 (+ 826us) server_negotiation.cc:299] Negotiation successful
0812 02:49:08.072083 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 02:49:08.075011 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:07.955222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53574 (local address 127.17.196.254:44969)
0812 02:49:07.955472 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:07.955480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:07.964936 (+ 9456us) server_negotiation.cc:407] Connection header received
0812 02:49:07.968768 (+ 3832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:07.968777 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:07.968891 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:07.968979 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:07.979261 (+ 10282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.980773 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.996102 (+ 15329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:07.997122 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:07.997274 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:08.038463 (+ 41189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:49:08.038477 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:49:08.038485 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:49:08.038563 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:49:08.044003 (+ 5440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:08.051437 (+ 7434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:08.051448 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:08.051454 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:08.051692 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:49:08.052274 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:49:08.052282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:49:08.052288 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:49:08.052855 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:49:08.053226 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:08.074078 (+ 20852us) server_negotiation.cc:299] Negotiation successful
0812 02:49:08.074489 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:49:08.111501 12359 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [term 1 FOLLOWER]: Refusing update from remote peer 08c511f8f7b4464a86f84106c629ef83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:49:08.112761 12434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:49:08.127300 12269 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [term 1 FOLLOWER]: Refusing update from remote peer 08c511f8f7b4464a86f84106c629ef83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:49:08.134259 12434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:49:08.155243 12398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08c511f8f7b4464a86f84106c629ef83. Latest consensus state: current_term: 1 leader_uuid: "08c511f8f7b4464a86f84106c629ef83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:08.155591 12398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:08.162518 12453 mvcc.cc:205] Tried to move back new op lower bound from 7317516075436716032 to 7317516074782011392. Current Snapshot: MvccSnapshot[applied={T|T < 7317516075436716032}]
I20260812 02:49:08.162348 12413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08c511f8f7b4464a86f84106c629ef83. Latest consensus state: current_term: 1 leader_uuid: "08c511f8f7b4464a86f84106c629ef83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:08.162727 12413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:08.169239 12413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08c511f8f7b4464a86f84106c629ef83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:08.169562 12413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:08.174994 12431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08c511f8f7b4464a86f84106c629ef83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:08.175309 12463 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 02:49:08.175359 12431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: This master's current role is: LEADER
I20260812 02:49:08.176783 12434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08c511f8f7b4464a86f84106c629ef83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:08.177100 12434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83 [sys.catalog]: This master's current role is: LEADER
I20260812 02:49:08.177898 12458 mvcc.cc:205] Tried to move back new op lower bound from 7317516075436716032 to 7317516074782011392. Current Snapshot: MvccSnapshot[applied={T|T < 7317516075436716032}]
I20260812 02:49:08.179656 12447 catalog_manager.cc:1347] Generated new cluster ID: 680a4c76993f49ecb1848a1980bc6b56
I20260812 02:49:08.179776 12447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:49:08.198378 12398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08c511f8f7b4464a86f84106c629ef83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c511f8f7b4464a86f84106c629ef83" member_type: VOTER last_known_addr { host: "127.17.196.254" port: 44969 } } peers { permanent_uuid: "2b863f588de742068ac2c92031de2a00" member_type: VOTER last_known_addr { host: "127.17.196.253" port: 39677 } } peers { permanent_uuid: "3db5dc9ed6fc451984e9fc01bce3af3d" member_type: VOTER last_known_addr { host: "127.17.196.252" port: 33915 } } }
I20260812 02:49:08.198657 12398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:49:08.200700 12447 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:49:08.204435 12447 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:49:08.224756 12447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08c511f8f7b4464a86f84106c629ef83: Generated new TSK 0
I20260812 02:49:08.226138 12447 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:49:08.391427 12396 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.365778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53580 (local address 127.17.196.254:44969)
0812 02:49:08.366015 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.366023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.388780 (+ 22757us) server_negotiation.cc:407] Connection header received
0812 02:49:08.389344 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.389351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.389453 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.389563 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.391119 (+ 1556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53580: BlockingRecv error: recv got EOF from 127.0.0.1:53580 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 02:49:08.394556 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.372753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.17.196.253:39677)
0812 02:49:08.372991 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.372997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.392334 (+ 19337us) server_negotiation.cc:407] Connection header received
0812 02:49:08.392874 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.392882 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.392973 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.393056 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.394286 (+ 1230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33942: BlockingRecv error: recv got EOF from 127.0.0.1:33942 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 02:49:08.398672 12392 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.395917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.17.196.252:33915)
0812 02:49:08.396145 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.396152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.396601 (+ 449us) server_negotiation.cc:407] Connection header received
0812 02:49:08.396880 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.396886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.396982 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.397056 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.398402 (+ 1346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35682: BlockingRecv error: recv got EOF from 127.0.0.1:35682 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 02:49:08.418704 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.415899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.17.196.253:39677)
0812 02:49:08.416123 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.416130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.416722 (+ 592us) server_negotiation.cc:407] Connection header received
0812 02:49:08.416990 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.416998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.417191 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.417337 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.418363 (+ 1026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33946: BlockingRecv error: recv got EOF from 127.0.0.1:33946 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260812 02:49:08.426321 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.423971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33956 (local address 127.17.196.253:39677)
0812 02:49:08.424188 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.424194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.424714 (+ 520us) server_negotiation.cc:407] Connection header received
0812 02:49:08.424910 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.424917 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.425027 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.425147 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.426062 (+ 915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33956: BlockingRecv error: recv got EOF from 127.0.0.1:33956 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 02:49:08.438479 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.435845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33972 (local address 127.17.196.253:39677)
0812 02:49:08.436061 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.436068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.436671 (+ 603us) server_negotiation.cc:407] Connection header received
0812 02:49:08.436903 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.436910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.437047 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.437166 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.438187 (+ 1021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33972: BlockingRecv error: recv got EOF from 127.0.0.1:33972 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 02:49:08.475857 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.471719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.17.196.253:39677)
0812 02:49:08.471961 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.471968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.474140 (+ 2172us) server_negotiation.cc:407] Connection header received
0812 02:49:08.474340 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.474346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.474456 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.474582 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.475534 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33974: BlockingRecv error: recv got EOF from 127.0.0.1:33974 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260812 02:49:08.491566 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.487907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33976 (local address 127.17.196.253:39677)
0812 02:49:08.488078 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.488082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.490038 (+ 1956us) server_negotiation.cc:407] Connection header received
0812 02:49:08.490306 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.490310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.490388 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.490506 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.491300 (+ 794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33976: BlockingRecv error: recv got EOF from 127.0.0.1:33976 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 02:49:08.514119 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.511697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.17.196.253:39677)
0812 02:49:08.511954 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.511961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.512427 (+ 466us) server_negotiation.cc:407] Connection header received
0812 02:49:08.512710 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.512719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.512847 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.512941 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.513870 (+ 929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33986: BlockingRecv error: recv got EOF from 127.0.0.1:33986 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 02:49:08.553074 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.550782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33998 (local address 127.17.196.253:39677)
0812 02:49:08.550996 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.551003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.551446 (+ 443us) server_negotiation.cc:407] Connection header received
0812 02:49:08.551741 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.551748 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.551859 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.551986 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.552790 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33998: BlockingRecv error: recv got EOF from 127.0.0.1:33998 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260812 02:49:08.624193 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.622005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.17.196.253:39677)
0812 02:49:08.622277 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.622284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.622710 (+ 426us) server_negotiation.cc:407] Connection header received
0812 02:49:08.622899 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.622905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.623011 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.623146 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.623953 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34010: BlockingRecv error: recv got EOF from 127.0.0.1:34010 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 02:49:08.748261 12410 catalog_manager.cc:1259] Loaded cluster ID: 680a4c76993f49ecb1848a1980bc6b56
I20260812 02:49:08.748373 12410 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: loading cluster ID for follower catalog manager: success
I20260812 02:49:08.751142 12410 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: acquiring CA information for follower catalog manager: success
I20260812 02:49:08.753235 12410 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3db5dc9ed6fc451984e9fc01bce3af3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 02:49:08.757956 12301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:49:08.755981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.17.196.253:39677)
0812 02:49:08.756276 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:08.756283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:08.756522 (+ 239us) server_negotiation.cc:407] Connection header received
0812 02:49:08.756716 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:08.756720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:08.756796 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:08.756888 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:49:08.757725 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34022: BlockingRecv error: recv got EOF from 127.0.0.1:34022 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
I20260812 02:49:08.828426 12428 catalog_manager.cc:1259] Loaded cluster ID: 680a4c76993f49ecb1848a1980bc6b56
I20260812 02:49:08.828532 12428 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: loading cluster ID for follower catalog manager: success
I20260812 02:49:08.831142 12428 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: acquiring CA information for follower catalog manager: success
I20260812 02:49:08.833076 12428 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b863f588de742068ac2c92031de2a00: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:49:09.025694 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:09.019528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.17.196.253:39677)
0812 02:49:09.019816 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:09.019823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:09.020069 (+ 246us) server_negotiation.cc:407] Connection header received
0812 02:49:09.020249 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:09.020254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:09.020417 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:09.020526 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:09.021034 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.021784 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.023435 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.023887 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.023986 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:09.024506 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:09.025045 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:09.025210 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:09.025263 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 02:49:09.025349 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260812 02:49:09.042052 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:09.035108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.17.196.252:33915)
0812 02:49:09.036477 (+ 1369us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:09.036483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:09.036507 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:49:09.036636 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:09.036640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:09.036800 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:09.037017 (+ 217us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:09.037534 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.038301 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.039655 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.040084 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.040233 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:09.040781 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:09.041332 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:09.041538 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:09.041641 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 02:49:09.041730 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1241,"thread_start_us":1109,"threads_started":1}
I20260812 02:49:09.059374 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:09.048685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.17.196.253:39677)
0812 02:49:09.048936 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:09.048942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:09.054214 (+ 5272us) server_negotiation.cc:407] Connection header received
0812 02:49:09.054437 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:09.054447 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:09.054523 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:09.054633 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:09.055120 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.055873 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.057386 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.057922 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.058084 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:09.058560 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:09.058690 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:09.058798 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:09.058965 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 02:49:09.059059 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 02:49:09.065586 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:09.050075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.17.196.252:33915)
0812 02:49:09.050235 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:09.050239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:09.059399 (+ 9160us) server_negotiation.cc:407] Connection header received
0812 02:49:09.059734 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:09.059738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:09.059819 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:09.059869 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:09.060380 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.061084 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.063121 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.063884 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.064022 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:09.064607 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:09.064708 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:09.064780 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:09.065007 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 02:49:09.065082 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 02:49:09.069365 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:49:09.048608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53592 (local address 127.17.196.254:44969)
0812 02:49:09.062664 (+ 14056us) server_negotiation.cc:206] Beginning negotiation
0812 02:49:09.062671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:49:09.062693 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:49:09.062825 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:49:09.062831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:49:09.063090 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:49:09.063969 (+ 879us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:49:09.064152 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.065140 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.067290 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:49:09.067714 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:49:09.067865 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:49:09.068388 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:49:09.068731 (+ 343us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:49:09.068856 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 02:49:09.068975 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 02:49:09.069067 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13601,"thread_start_us":13507,"threads_started":1}
I20260812 02:49:09.074772 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 12130
I20260812 02:49:09.075968 12207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:49:09.188827 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 12130
W20260812 02:49:09.216216 12327 connection.cc:535] server connection from 127.0.0.1:35658 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:49:09.217068 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 12222
I20260812 02:49:09.218379 12290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:49:09.332672 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 12222
I20260812 02:49:09.361613 18195 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu pid 12306
I20260812 02:49:09.363003 12383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:49:09.477721 18195 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDYHVZq/build/asan/bin/kudu with pid 12306
2026-08-12T02:49:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5280 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10790 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (401860 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST